Diss. ETH No.

14738

Quadratic Programming in
Geometric Optimization:
Theory, Implementation, and
Applications

Sven Schönherr

2002
Diss. ETH No. 14738

Quadratic Programming in
Geometric Optimization:
Theory, Implementation, and
Applications
Dissertation

submitted to the
Swiss Federal Institute of Technology
Zurich, Switzerland

for the degree of

Doctor of Technical Sciences

presented by
Sven Schönherr
Diplom-Mathematiker, Freie Universität Berlin, Germany
born on March 17th , 1968, in Berlin, Germany
citizen of Germany

accepted on the recommendation of

Prof. Dr. Emo Welzl, examiner
Prof. Dr. Thomas Liebling, co-examiner
Dr. Bernd Gärtner, co-examiner

2002
 Für Marc-Daniel,
unseren Sonnenschein
Acknowledgments

I want to thank all people who have contributed by some means or other
to the success of this thesis.

First and foremost among these is Bernd Gärtner, who supported and
guided me during the whole time of my studies. He gave me valuable
insights in the area of geometric optimization and always kept me on
the right track in countless discussions. The subject of this thesis was
proposed by him, and, without his ongoing encouragement and strong
support, the completion of this work would not have been possible. My
sincere thanks go to him.

I am deeply indebted to Emo Welzl for giving me the opportunity to

work in his research groups in Berlin and Zurich. I learned a lot from
him, and his elegant way of presenting mathematical problems and their
solutions has always inspired me.

I am very grateful to Thomas Liebling for agreeing to be a co-examiner.

His valuable comments helped improving the presentation.

The collegial atmosphere of the research group at ETH made my time

in Zurich a big pleasure. Special thanks go to Christoph Ambühl, Udo
Adamy, Falk Tschirschnitz, Zsuzsa Lipták, and Brigitte Kröll for the
opportunities to balance work and leisure.

Let me express my best thanks to Helmut Alt, who enabled me to

continue my scientific work in Berlin after Emo Welzl moved to Zurich.
I owe many thanks to my colleagues at Freie Universität Berlin for
the pleasurable atmosphere, especially to Christian Knauer, Hannah
Heinrich, Alexander Wolff, Frank Hoffmann, Klaus Kriegel, and Carola
Wenk.

One of the highlights during my studies was the participation in the

development of Cgal. I am grateful to the whole Cgal team for the
very interesting and successful cooperation. My special thanks go to
Andreas Fabri, Geert-Jan Giezemann, Lutz Kettner, and Stefan Schirra
for the intensive discussions on C++ design and software engineering.
I really enjoyed the cooperation with the partners of the two applica-
tion projects. In particular, Hans-Jakob Lüthi, Artan Boriçi, Claude
Diderich, Phillipe Schiltknecht, and Sébastien Barbey in the portfolio
optimization project, and Udo Warschewske, Arno Schleich, and Wlad
Oganowski in the stylus calibration project.

The constant support of my family has been fundamental for the success
of this thesis. I thank them all, especially my wife, Martina.
Abstract

Many geometric optimization problems can be formulated as instances

of linear or quadratic programming. Examples are the polytope distance
problem, the smallest enclosing ball problem, the smallest enclosing
annulus problem, and the optimal separating hyperplane problem. The
resulting linear and quadratic programs share the property, that either
the number of variables or the number of constraints is small. This is
different from the usual setting in operations research, where both the
number of variables and the number of constraints are large, while the
matrix representing the constraints is sparse.

We present a solver for quadratic programming problems, which is tuned

for applications in computational geometry. The solver implements
a generalization of the simplex method to quadratic programs. Unlike
existing solvers, it is efficient if the problem is dense and has a small
number of variables or a small number of constraints.

A natural generalization of the smallest enclosing ball is the smallest

enclosing ellipsoid. This problem is an instance of convex programming
which cannot be formulated as a quadratic program. We derive explicit
formulae for the primitive operations of a randomized method by Welzl
in dimension d = 2. Compared to previous solutions, our formulae are
simpler and faster to evaluate, and they only contain rational expres-
sions, allowing for an exact solution.

In recent years the focus of interest in the field of computational geome-

try shifted from the design to practical aspects of the developed algo-
rithms and data structures. One prominent exponent of the new focus
is Cgal, the Computational Geometry Algorithms Library. It is a soft-
ware library of geometric objects, data structures and algorithms writ-
ten in C++. We implemented our new quadratic programming solver
as part of Cgal. Based on it, we realized solutions for the first three
geometric optimization problems mentioned above. The implementa-
tions are carefully designed to be efficient and easy to use at the same
time. The distinguishing features of the solver are almost no overhead
when used to solve linear programming problems, user-specifiable pro-
gram representations, and a combination of exact and floating point
arithmetic. This results in a provably correct and usually very efficient
implementation, which is underpinned by the experimental results.

The practical usefulness is shown by two applications in financial busi-

ness and medicine, where our method solved ‘real-world’ problems. The
quadratic programming solver was used to solve a large portfolio optimi-
zation problem as part of a joint project with a big Swiss bank (actually,
we showed that the problem was infeasible). In another cooperation
with a German company developing medical navigation systems, the
smallest enclosing annulus implementation was used to calibrate the tip
of medical probes.

We showed in this thesis, that quadratic programming can be used for

efficient solutions of some geometric optimization problems and their
applications. We developed a solver for linear and quadratic programs,
which is tuned for the problems we were interested in. Its realization in
Cgal gives always correct results and is very efficient in most practical
cases.
Kurzfassung

Viele geometrische Optimierungsprobleme können als Instanzen von

Linearem oder Quadratischem Programmieren formuliert werden. Bei-
spiele sind das Problem des Abstands zweier Polytope, das Problem der
kleinsten umschliessenden Kugel, das Problem des kleinsten umschlies-
senden Kugelrings und das Problem der optimal separierenden Hyper-
ebene. Die resultierenden Linearen und Quadratischen Programme
haben als gemeinsame Eigenschaft, dass entweder die Anzahl der Unbe-
kannten oder die Anzahl der Nebenbedingungen klein ist. Dies unter-
scheidet sie vom üblichen Szenario im Bereich Operations Research, wo
die Anzahl der Unbekannten und die Anzahl der Nebenbedingungen
gross sind, während die Matrix der Nebenbedingungen dünn besetzt
ist.

Wir präsentieren einen Löser für Quadratisches Programmieren, der auf

Anwendungen in der Algorithmischen Geometrie zugeschnitten ist. Der
Löser ist eine Verallgemeinerung der Simplex Methode auf Quadratische
Programme. Im Gegensatz zu bereits existieren Lösern ist er effizient,
wenn das Problem dicht besetzt ist und eine kleine Anzahl von Unbe-
kannten oder eine kleine Anzahl von Nebenbedingungen hat.

Eine natürliche Verallgemeinerung der kleinsten umschliessenden Kugel

ist das kleinste umschliessende Ellipsoid.Dieses Problem ist eine Instanz
von Konvexem Programmieren, welche sich nicht als Quadratisches Pro-
gramm formulieren lässt. Wir leiten explizite Formeln für die Basisope-
rationen eines randomisierten Algorithmus von Welzl in zwei Dimen-
sionen her. Verglichen mit vorherigen Lösungen sind unsere Formeln
einfacher und schneller auswertbar. Sie enthalten nur rationale Aus-
drücke und ermöglichen so eine exakte Lösung.

In den letzten Jahren hat sich der Interessenschwerpunkt in der Algo-

rithmischen Geometrie vom Design zu den praktischen Aspekten der
entwickelten Algorithmen und Datenstrukturen verschoben. Ein pro-
minenter Vertreter des neuen Schwerpunkts ist Cgal, die Computa-
tional Geometry Algorithms Library. Dies ist eine in C++ geschrie-
bene Softwarebibliothek von geometrischen Objekten, Datenstrukturen
und Algorithmen. Wir haben unseren neuen Löser für Quadratisches
Programmieren als Teil von Cgal implementiert. Darauf aufbauend
haben wir Lösungen für die ersten drei der oben genannten geome-
trischen Optimierungsprobleme realisiert. Die Implementierungen sind
sorgfältig entworfen, um zugleich effizient und einfach benutzbar zu sein.
Die herausragenden Merkmale des Lösers sind fast keine Zusatzkosten
beim Lösen von Linearen Programmen, eine durch den Benutzer spe-
zifizierbare Repräsentierung des Problems und eine Kombination aus
Fliesskomma- und exakter Arithmetik. Daraus ergibt sich eine beweis-
bar korrekte und in den meisten Fällen sehr effiziente Implementierung.
Dies wird durch die experimentellen Ergebnisse unterstrichen.

Die Praxistauglichkeit wird anhand zweier Applikationen in den Berei-

chen Finanzen und Medizin gezeigt, wo unsere Methode Probleme aus
der realen Welt“ gelöst hat. Der Löser für Quadratische Programme
”
wurde zum Lösen eines grossen Wertpapier-Optimierungsproblems ver-
wendet (genauer gesagt haben wir gezeigt, dass das Problem keine
Lösung besass). Im Rahmen einer Zusammenarbeit mit einer deutschen
Firma, die ein medizinisches Navigationssystem entwickelt, haben wir
die Lösung zur Berechnung des kleinsten umschliessenden Kugelrings
zum Kalibrieren der Spitze von medizinischen Instrumenten eingesetzt.

Wir haben in dieser Arbeit gezeigt, dass Quadratisches Programmieren

zur effizienten Lösung von einigen geometrischen Optimierungsproble-
men und ihren Anwendungen verwendet werden kann. Wir haben einen
Löser für Quadratische Programme entwickelt, der auf die betrachteten
Probleme zugeschnitten ist. Seine Realisierung in Cgal liefert immer
korrekte Ergebnisse und ist in den meisten praktischen Fällen effizient.
Contents

0 Introduction 1
0.1 Geometric Optimization . . . . . . . . . . . . . . . . . . 3
0.2 Statement of Results . . . . . . . . . . . . . . . . . . . . 9
0.3 Outline of the Thesis . . . . . . . . . . . . . . . . . . . . 10

Part I: Theory

1 Linear and Quadratic Programming 15

1.1 Standard Form . . . . . . . . . . . . . . . . . . . . . . . 16
1.2 Simplex Method . . . . . . . . . . . . . . . . . . . . . . 17

2 A Simplex-Like Quadratic Programming Algorithm 19

2.1 Karush-Kuhn-Tucker Conditions . . . . . . . . . . . . . 20
2.2 Basic Solutions . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Pivot Step . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Pricing . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 Ratio Test . . . . . . . . . . . . . . . . . . . . . . 25
2.3.3 Update . . . . . . . . . . . . . . . . . . . . . . . 31
2.4 Inequality Constraints . . . . . . . . . . . . . . . . . . . 31
2.5 Degeneracies . . . . . . . . . . . . . . . . . . . . . . . . 34
2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 35
3 Geometric Optimization Problems 37
3.1 Polytope Distance . . . . . . . . . . . . . . . . . . . . . 37
3.2 Smallest Enclosing Ball . . . . . . . . . . . . . . . . . . 39
3.3 Smallest Enclosing Annulus . . . . . . . . . . . . . . . . 42
3.4 Optimal Separating Hyperplane . . . . . . . . . . . . . . 43
3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 44

4 A Non-Quadratic Geometric Optimization Problem 45

4.1 Smallest Enclosing Ellipsoid . . . . . . . . . . . . . . . . 45
4.2 Welzl’s Method . . . . . . . . . . . . . . . . . . . . . . . 47
4.3 Exact Primitives in the Plane . . . . . . . . . . . . . . . 50
4.3.1 Conics . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.2 In-ellipse Test, |R| = 3 . . . . . . . . . . . . . . . 52
4.3.3 In-ellipse Test, |R| = 4 . . . . . . . . . . . . . . . 52
4.3.4 In-ellipse Test, |R| = 5 . . . . . . . . . . . . . . . 58
4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 58

Part II: Implementation

5 Cgal, the Computational Geometry Algorithms Library 63

5.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . 64
5.2 Generic Programming . . . . . . . . . . . . . . . . . . . 65
5.3 Library Structure . . . . . . . . . . . . . . . . . . . . . . 69
5.4 Geometric Kernel . . . . . . . . . . . . . . . . . . . . . . 71
5.5 Basic Library . . . . . . . . . . . . . . . . . . . . . . . . 74
5.5.1 Generic Data Structures . . . . . . . . . . . . . . 74
5.5.2 Generic Algorithms . . . . . . . . . . . . . . . . . 75
5.5.3 Traits Classes . . . . . . . . . . . . . . . . . . . . 76
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 79
6 A Quadratic Programming Engine 81
6.1 Design Goals . . . . . . . . . . . . . . . . . . . . . . . . 81
6.2 Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.2.1 Access to Original Problem . . . . . . . . . . . . 83
6.2.2 Tags for Special Properties . . . . . . . . . . . . 84
6.2.3 Exact Arithmetic . . . . . . . . . . . . . . . . . . 86
6.2.4 Symbolic Perturbation . . . . . . . . . . . . . . . 86
6.3 Basis Inverse . . . . . . . . . . . . . . . . . . . . . . . . 87
6.3.1 Rational Representation . . . . . . . . . . . . . . 88
6.3.2 Updates . . . . . . . . . . . . . . . . . . . . . . . 89
6.3.3 Implementation . . . . . . . . . . . . . . . . . . . 94
6.4 Pricing Strategies . . . . . . . . . . . . . . . . . . . . . . 95
6.4.1 Partial Pricing . . . . . . . . . . . . . . . . . . . 97
6.4.2 Filtered Pricing . . . . . . . . . . . . . . . . . . . 98
6.4.3 Implementations . . . . . . . . . . . . . . . . . . 101
6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 105

7 Geometric Optimization Problems in Cgal 107

7.1 Polytope Distance . . . . . . . . . . . . . . . . . . . . . 107
7.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . 107
7.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . 108
7.1.3 Implementation . . . . . . . . . . . . . . . . . . . 108
7.2 Smallest Enclosing Ball . . . . . . . . . . . . . . . . . . 111
7.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . 111
7.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . 111
7.2.3 Implementation . . . . . . . . . . . . . . . . . . . 112
7.3 Smallest Enclosing Annulus . . . . . . . . . . . . . . . . 113
7.3.1 Definition . . . . . . . . . . . . . . . . . . . . . . 113
7.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . 113
7.3.3 Implementation . . . . . . . . . . . . . . . . . . . 114
7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 115
8 Experimental Results 117
8.1 Test Environment . . . . . . . . . . . . . . . . . . . . . . 117
8.2 Polytope Distance . . . . . . . . . . . . . . . . . . . . . 119
8.3 Smallest Enclosing Ball . . . . . . . . . . . . . . . . . . 119
8.4 Smallest Enclosing Annulus . . . . . . . . . . . . . . . . 123
8.5 Smallest Enclosing Ellipse . . . . . . . . . . . . . . . . . 126
8.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 127

Part III: Applications

9 Two Applications 131

9.1 Portfolio Optimization . . . . . . . . . . . . . . . . . . . 131
9.1.1 Application Background . . . . . . . . . . . . . . 131
9.1.2 Problem Description . . . . . . . . . . . . . . . . 132
9.1.3 Results . . . . . . . . . . . . . . . . . . . . . . . 132
9.2 Stylus Calibration . . . . . . . . . . . . . . . . . . . . . 133
9.2.1 Application Background . . . . . . . . . . . . . . 133
9.2.2 Problem Description . . . . . . . . . . . . . . . . 134
9.2.3 Solution . . . . . . . . . . . . . . . . . . . . . . . 135
9.2.4 Experimental Results . . . . . . . . . . . . . . . 136

Bibliography 139

Index 149
Chapter 0

Introduction

Computational Geometry is the subfield of algorithm design that deals

with the design and analysis of solutions for problems involving geo-
metric objects. Over the past two decades an enormous progress was
made in the development of efficient geometric algorithms and data
structures. However, many of these techniques have not found their
way into practice yet. An important cause for this is the fact that the
correct implementation of even the simplest of these algorithms can be
a notoriously difficult task [66].

There are two particular aspects that need to be dealt with to close
the gap between the theoretical results of computational geometry and
practical implementations [86], namely the precision problem and the
degeneracy problem.

Theoretical papers assume exact arithmetic with real numbers. The cor-
rectness proofs of the algorithms rely on the exact computations, thus
replacing exact arithmetic by imprecise built-in floating point arithmetic
does not work in general. Geometric algorithms in particular are sensi-
tive to rounding errors since numerical data and control flow decisions
have usually strong interrelations. The numerical problems may destroy
the combinatorial or geometric consistency of the algorithm, thus result-
ing in a program crash, in an infinite loop, or in unpredictable erroneous
output.
2 Chapter 0. Introduction

Often, theoretical papers exclude degenerate configurations in the input.

Typically, these degeneracies are specific to the problem and would in-
volve the treatment of special cases in the algorithm. Simple examples of
configurations considered as degenerate are duplicate points in a point
set or three lines intersecting in one point. For some problems, it is
not difficult to handle the degeneracies, but for others the special case
treatment distracts from the solution of the general problem and it can
amount to a considerable fraction of the coding effort.

In theory, this approach of excluding degeneracies from consideration is

justified by the argument that degenerate cases are very rare in the set
of all possible inputs over the real numbers, i.e. they are very unlikely
if the input set is randomly chosen over the real numbers. Another
argument is that it is first of all important to understand the general
case before treating special cases.

In practice, however, degenerate inputs do occur quite frequently. For

instance, the coordinates of the geometric objects may not be randomly
chosen over the real numbers, but lie on a grid. For example, they
may be created by clicking in a window in a graphical user interface.
In some applications, what are called degeneracies are even high-valued
design criteria. In architecture, for example, features of buildings do
align on purpose. As a consequence, practical implementations usually
must address the handling of degeneracies.

Besides the precision problem and the degeneracy problem, advanced

algorithms bring about the additional difficulty that they are frequently
hard to understand and hard to code.

In recent years the focus of interest in the field of computational geome-

try shifted from the design to practical aspects of the developed algo-
rithms and data structures. One prominent exponent of the new focus
is the Cgal1 project [78]. The mission statement of the project (and
its successors Galia2 and Ecg3 ) is to

make the large body of geometric algorithms developed in

the field of computational geometry available for industrial
application.
1 Constructing a Geometric Algorithms Library
2 Geometric Algorithms for Industrial Applications
3 Effective Computational Geometry for Curves and Surfaces
0.1. Geometric Optimization 3

The key tool in reaching this goal is Cgal, the Computational Geometry
Algorithms Library. It was developed during the first two projects by
several universities and research institutes in Europe and Israel. Since
then, the library is (and will be in the future) actively maintained and
extended.

I joined the Cgal team in January 1995. As one of the five members
of the kernel design group [30], I influenced the design of the geometric
kernel and the overall design of the library. My contributions to the
basic library lie in the area of geometric optimization. In particular,
I implemented two versions of Welzl’s algorithm [105] for computing
the smallest enclosing circle and the smallest enclosing ellipse, respec-
tively, of a finite set of points in the plane. The latter implementation
uses the exact primitives derived in Chapter 4. It is joint work with
Bernd Gärtner, who wrote the underlying conic class [43, 44]. Further
contributions are implementations of geometric optimization problems
based on the new quadratic programming solver presented in this thesis
as well as the implementation of the solver itself.

0.1 Geometric Optimization

Many geometric optimization problems can be formulated as instances

of linear programming (LP) or quadratic programming (QP), where
either the number of variables n or the number of constraints m is
small. LP is concerned with minimizing a linear function subject to
linear (in)equality constraints, while QP deals with convex quadratic
objective functions of the form

cT x + xTD x,

with D being a positive semi-definite matrix. Examples are the follow-

ing:

Polytope distance. Given two polytopes P and Q, defined by a total

of k vertices (or k halfspaces) in d-dimensional space, find two points
p ∈ P and q ∈ Q with smallest Euclidean distance. For fixed d, an
O(k) solution follows from the fact that this problem can be formu-
4 Chapter 0. Introduction

lated as an LP-type problem4 in the sense of [64]. It has explicitly been

addressed by Wolfe for the special case where one polytope is defined
by a single point [107] and by Sekitani and Yamamoto in the general
case [94]. The latter algorithm applies the technique Welzl has used
for the smallest enclosing ball problem [105] (see below), while the for-
mer works in a simplex-like fashion. In fact, when specialized to the
situation in Wolfe’s paper, our algorithm for solving QP problems is an
implementation of his method. A related problem, namely that of find-
ing the nearest point in a simplicial cone, has been studied by Murty
and Fathi in [71]. The polytope distance problem – mostly occurring in
collision detection – immediately fits into the QP framework.

Smallest enclosing ball. Given k points in d-dimensional space, find

the ball of smallest volume containing all the points. This is a classical
problem of computational geometry, with many applications, for exam-
ple in bounding volume heuristics. For fixed dimension d, O(k) algo-
rithms are known [105], and efficient implementations exist [40, 19].
The smallest enclosing ball problem is a special case of a certain class
of optimal containment problems [27, 35], which can be formulated as
QP problems. A concrete formulation as a quadratic program together
with a proof of its correctness is given in Chapter 3.

Smallest enclosing annulus. Given k points in d-dimensional space,

find the annulus (region between concentric spheres with radii R and r,
R ≥ r) that contains the points and minimizes the difference R2 − r2 .
For d = 2, this is the annulus of smallest area, and in any dimension,
the optimal annulus can be used to test whether the input points lie
approximately on a sphere. Roundness tests in general have received
some attention recently [60]. The problem is also of theoretical impor-
tance, because it can be used in an approximation algorithm for the
minimum-width annulus, a much harder problem [2]. The smallest
4 The abstract class of LP-type problems as introduced by Sharir and Welzl [64]

is a generalization of the class of LP problems. An LP-type problem is a pair (H, w)

with H being a set of ‘constraints’ and w being an ‘objective function’, where each
subset G ⊆ H gets assigned a value w(G) with the following properties: Let F ⊆ G,
then w(F ) ≤ w(G) (monotonicity) and if w(F ) = W (G) and w(G) < w(G∪{h}) for
some h ∈ H \G, then also w(F ) < w(F ∪{h}) (locality). A basis of G ⊆ H is an
inclusion-minimal subset B of G such that w(B) = w(G). To solve the problem
means to find w(H) and a basis of H. A linear program is a special case of an
LP-type problem.
0.1. Geometric Optimization 5

enclosing annulus problem is an LP problem, but it can of course also be

considered as QP with a ‘degenerate’ objective function, having D = 0.

Optimal separating hyperplane. Given two point sets P and Q

with a total of k points in d-dimensional space, test whether they can
be separated by a hyperplane, and if so, find a separating hyperplane
that maximizes the distance to the nearest point. The separation itself
can be done by LP, but finding the optimal hyperplane is QP. The
problem can be reduced to the polytope distance problem.

It is common to all four problems mentioned above, that d is usually

small compared to k, often even constant (d = 2, 3). In order to solve
them efficiently, this has to be taken into account. Like in case of LP,
existing solvers for QP are not tuned for this scenario. They usually
work in the operations research setting, where both the number of con-
straints m and the number of variables n are large, but the matrix
representing the constraints is sparse. Of course, such solvers can be
used to address problems in our scenario, but a performance penalty is
unavoidable in this case.

The scenario for which we developed the QP solver is the one where
min(n, m) is small. In the geometric optimization problems, this value
is closely related to the dimension of the space the problem lives in. The
value max(n, m), on the other hand, may be very large — it usually
comes from the number of objects (points, halfspaces, etc.) that define
the problem. Moreover, we assume no sparsity conditions, i.e. both
matrices A and D may be dense. Namely, the geometric optimization
problems we want to handle typically exhibit a dense structure; when
we deal with point sets in Euclidean space, for example, the concept
of sparsity can even be meaningless. Many properties of point sets are
translation invariant, in which case coordinates with value zero play no
special role at all.

Let us briefly discuss why ‘standard’ solvers (addressing the case where
both m and n may be large and the problem is sparse) cannot efficiently
be used in our scenario.

First of all, if n is large and the objective matrix D is dense, the prob-
lem cannot even be entered into solvers which require the Θ(n2 ) nonzero
entries of D to be explicitly given. This situation for example occurs
6 Chapter 0. Introduction

in connection with the smallest enclosing ball problem. There is an

alternative formulation in which D is sparse, but in return, the con-
straint matrix A will get larger. Our solver can handle implicitly given
matrices D, and if m is small, only few entries will ever be evaluated.

Furthermore, the performance of standard solvers in practice crucially

depends on the number of nonzero entries in the problem description,
rather than on n and m (which is exactly what one wants for large,
sparse problems). In our scenario, this is inappropriate, because in
the dense case, no advantage is taken from the fact that one of the
parameters is small.

It is well-known that the simplex method for LP can be generalized to

deal with QP problems. For this one usually blows up the size of the
constraint matrix in such a way that both the number of variables and
the number of constraints become large [106]. In our scenario, this would
mean to give away the crucial advantage of having one parameter small.
But, if care is taken, the simplex method can smoothly be generalized
to QP, where the blow-up is limited by the rank of D, rather than its
size. This rank is usually closely related to min(n, m) in the problems
we are interested in, even if max(n, m) is large.

A simplex-like method has another advantage: it generates basic solu-

tions. In the smallest enclosing ball problem, for example, this means
that not only the optimal radius, but also the input points that deter-
mine the optimal ball are computed. In contrast, QP problems are often
solved using interior point methods5 and to obtain basic solutions from
that requires additional effort.

In practice, the simplex-like approach is polynomial in both parameters

n and m of the QP problem. This means, we can solve the concrete
geometric optimization problems mentioned above even in moderately
high dimensions (50 ≤ d ≤ 300, depending on the problem). In partic-
ular, for the smallest enclosing ball problem this substantially extends
the range of feasible dimensions that can be handled by computational
geometry codes. Interior point codes (like CPLEX’s QP solver) can go
much higher, so if that is required, they are the ones to choose (with
the disadvantages mentioned above).
5 for example, CPLEX’s quadratic programming solver does this6
6 CPLEX is a trademark of CPLEX Optimization Inc.
0.1. Geometric Optimization 7

A major issue any serious LP or QP solver has to address is numerical

accuracy [39]. On the one hand the program must not crash due to
numerical problems, on the other hand the computed result should be
correct. What correctness means, and what the best way is to achieve it,
depends on the application. The crucial point is that general purpose
LP and QP solvers must be able to handle any problem but may take
advantage of inputs they typically expect. Gärtner [39] classifies the
existing solvers into two categories: Either the ‘may’ part is ignored,
i.e. the solver is expecting the worst, or the ‘must’ part is neglected,
i.e. the solver is hoping for the best.

Expecting the worst. This strategy avoids numerical errors com-

pletely by using an exact number type in all arithmetic operations,
with the disadvantage of a big performance penalty. It is too pessimistic
in the sense that floating point operations are assumed to give wrong
results all the time, while in practice they work fine most of the time.
This approach is implemented, for example, in the LP solvers that are
part of vertex-enumeration codes [5, 36].

Hoping for the best. This strategy performs all numerical opera-
tions purely with floating point arithmetic. Although this is fast and
will produce the correct result in most cases, it fails on some problems.
This approach can be seen as too optimistic in the sense that all prob-
lems are assumed to be well-behaved, where in practice only most of
them are. It is used, for example, in the state-of-the-art solver CPLEX.

Summarizing, expecting the worst is always correct but also always slow,
while hoping for the best is always fast and usually correct. For our
scenario, we use a mixed strategy expecting the best and coping with the
worst by combining floating point and exact arithmetic as suggested
in [39]. This approach has the advantage of being always correct and
usually fast.

The following geometric optimization problem is a natural generaliza-

tion of the smallest enclosing ball problem, but it cannot be formulated
as a quadratic program.

Smallest enclosing ellipsoid. The problem of finding the smallest

enclosing ellipsoid of a k-point set P in d-space is an instance of con-
8 Chapter 0. Introduction

vex programming. Several algorithms for computing the minimal ellip-

soid have been proposed. On the one hand, there are iterative meth-
ods which employ standard optimization techniques (such as gradient
descent), adapted to the problem [102, 95]. These algorithms usually
work on a dual problem, known as D-optimal design [101]. On the other
hand, there are finite methods which find the desired ellipsoid within a
bounded number of steps. For fixed d, the algorithm of Post [81] has
complexity O(k 2 ). An optimal deterministic O(k) algorithm has been
given by Dyer [26], randomized O(k) methods are due to Adler and
Shamir [1] and Welzl [105]. Since the problem is LP-type in the sense
of [64], generic algorithms for this class of problems can be applied as
well, see [46]. In any case, the runtime dependence on d is exponential.
A method for the case d = 2, without time analysis, has been developed
by Silverman and Titterington [97].

All finite methods have the property that actual work is done only for
problem instances whose size is bounded by a function in d. Assuming
that d is constant, such instances can be solved in constant time. How-
ever, as far as explicit formulae for these primitive operations have been
given — which is the case only for d = 2 — they are quite complicated
and rely on solving third-degree polynomials [97, 80, 88].

In Chapter 4 we derive explicit formulae for the primitive operations

of Welzl’s randomized method [105] in dimension d = 2. Compared to
previous ones, these formulae are simpler and faster to evaluate, and
they only contain rational expressions, allowing for an exact solution.

The unique ellipsoid of smallest volume enclosing a compact set P in

d-space (also known as the Löwner-John ellipsoid of P [56]) has appeal-
ing mathematical properties which make it theoretically interesting and
practically useful. In typical applications, a complicated body needs
to be covered by a simple one of similar shape and volume, in order
to simplify certain tests. For convex bodies (e.g. the convex hull of a
finite point set), the smallest enclosing ellipsoid – unlike the isothetic
bounding box or the smallest enclosing sphere – guarantees a volume
approximation ratio that is independent of the shape of the covered
body. This is implied by the following property, first proved by John
(who also established existence and uniqueness) [56]: if the smallest
enclosing ellipsoid of a compact convex body K is scaled about its cen-
ter with factor 1/d, the resulting ellipsoid lies completely inside K.
0.2. Statement of Results 9

0.2 Statement of Results

The main results of this thesis can be summarized as follows.

• We developed a solver for quadratic programming problems, which

is tuned for applications in computational geometry. The solver
implements a generalization of the simplex method to quadratic
programs. Unlike existing solvers, it is efficient if the problem
is dense and has few variables or few constraints. Our method is
based on a preliminary version proposed by Gärtner [38], which we
worked out to a mathematically correct and practically efficient
algorithm.
• For the convex programming problem of computing the smallest
enclosing ellipse of a planar point set, we describe (together with
Gärtner [42]) primitives for a randomized algorithm of Welzl [105],
leading to a rational arithmetic solution. An implementation of
Welzl’s method in combination with our primitives is realized in
Cgal. It is, to our knowledge, the only available implementation
for the smallest enclosing ellipse problem that is based on exact
arithmetic, thus guaranteeing the correctness of the solution.
• We implemented our new quadratic programming solver in C++.
The implementation is carefully designed to be efficient and easy
to use at the same time. The distinguishing features are a com-
bination of exact and floating point arithmetic, user-specifiable
problem representations, and almost no overhead when used as
an LP solver. We used generic programming with class templates,
compile time tags, and iterators to reach our design goals. The
flexibility for the pricing strategy was achieved through an object-
oriented design with a base class and virtual functions. The hybrid
arithmetic approach in the partial filtered pricing scheme leads to
a considerable speed-up. The practical usefulness is underpinned
by the experimental results.
• We successfully applied our method to solve ‘real-world’ problems.
As part of a joint project with a big Swiss bank, our quadratic pro-
gramming solver was used to solve a large portfolio optimization
problem (actually, we showed that the problem was infeasible). In
another cooperation with a German company developing medical
navigation systems, the smallest enclosing annulus implementa-
tion was used to calibrate the tip of medical probes.
10 Chapter 0. Introduction

0.3 Outline of the Thesis

The thesis is subdivided into three parts. The first part contains the
theoretical background for the implementation work described in the
second part. The third part presents two applications.

Part I. We start in Chapter 1 with the definition of linear and quad-

ratic programs and give a brief outline of the classical simplex method.
Chapter 2 describes our new algorithm for solving quadratic program-
ming problems. We derive optimality criteria for quadratic programs
and explain the details of the extended pivot step. Furthermore, we
show an efficient way of handling inequality constraints and how to cope
with degeneracies. In Chapter 3 we present four geometric optimization
problems that can be solved by quadratic programming. Explicit for-
mulations of the corresponding quadratic or linear programs are given.
One of these problems, namely the smallest enclosing ball problem, has a
natural generalization to ellipsoids. Chapter 4 describes exact primitives
for solving the smallest enclosing ellipsoid problem in two dimensions.

Part II. Cgal, the Computational Geometry Algorithms Library is

introduced in Chapter 5. We review related work on geometric software
and precursors of Cgal. A section on generic programming is followed
by an overview of the library structure. More detailed descriptions
are given of the design of the geometric kernel and the basic library.
Chapter 6 describes a C++ implementation of our quadratic program-
ming solver. We state our design goals and show how we achieved them.
This is done by presenting the distinguishing implementation details and
small chunks of the code. In Chapter 7 we describe the three geometric
quadratic programming problems that are already available in Cgal.
We define the problems, show how to solve them, and sketch some
important implementation details. These implementations were tested
on different input sets and compared to other mathematical program-
ming implementations as well as to dedicated geometric algorithms. The
experimental results are presented in Chapter 8.

Part III. Chapter 9 presents two applications. The first one is a

financial application of our quadratic programming solver. It is used
to solve portfolio optimization problems. The second one is a medical
application, where the tip of a medical probe, a so called stylus, has
to be calibrated. We use the solution of the smallest enclosing annulus
problem in this application.
Part I

Theory
Chapter 1

Linear and Quadratic

Programming

Linear programming (LP) is the problem of minimizing a linear function

in n variables, subject to m linear (in)equality constraints over the vari-
ables. In addition, the variables may have to lie between prespecified
bounds. In this general formulation, a linear program can be written as

(LP) minimize cT x
subject to A x S b (1.1)
` ≤ x ≤ u,

where A is an m×n-matrix, b an m-vector, c an n-vector, and `, u are

n-vectors of bounds (values −∞ and ∞ may occur). The symbol ‘S’ in-
dicates that any of the m order relations it stands for can independently
be ‘≤’, ‘=’, or ‘≥’.

Generalizing LP to quadratic programming (QP) means replacing the

linear function in (1.1) by the convex quadratic function

cT x + xTD x ,

resulting in the quadratic program

16 Chapter 1. Linear and Quadratic Programming

(QP) minimize cT x + xTD x

subject to A x S b (1.2)
` ≤ x ≤ u.

Here, D is a positive semi-definite n×n-matrix1 . If D = 0, we obtain a

linear program as a special case of QP.

If a vector x∗ = (x∗1 , . . . , x∗n )T exists that satisfies all constraints, the

problem is called feasible and x∗ is a feasible solution, otherwise the
problem is called infeasible. If the objective function f (x) = cTx + xTDx
is bounded from below on the set of feasible solutions x∗ , the problem
is called bounded, otherwise unbounded. If the problem is both feasible
and bounded, the objective function assumes its unique minimum value
for some (not necessarily unique) optimal feasible solution x∗ . Solving
a quadratic program means either finding an optimal solution x∗ if one
exists, else proving that no feasible solution exists or that there is no
bounded optimum.

1.1 Standard Form

For the description, we consider QP in standard form, given as

(QP) minimize cT x + xTD x

subject to A x = b (1.3)
x ≥ 0,

where the number of variables n is at least as large as the number of

equality constraints m. As in the simplex method for LP described
in the next section, explicit bounds ` ≤ x ≤ u in (1.2) can smoothly
be integrated, and if inequality constraints occur, they can be turned
into equalities by introducing slack variables. This means, if m is small
in (1.2), we may without loss of generality assume that the quadratic
program is in standard form, having only few equality constraints.

If m  n in (1.2), the problem contains many inequality constraints

in all interesting cases; however, turning them into equalities gener-
ates a large number of slack variables, so that it is no longer true that
1 i.e. xTD x ≥ 0 holds for all x.
1.2. Simplex Method 17

the resulting standard form has few equality constraints. The crucial
observation here is that the former inequalities can be treated implic-
itly; intuitively, we ‘trade them in’ for the new slack variables, moving
the problem’s complexity from the constraints to the variables, after
which we are basically back to a quadratic program in standard form
with m ≤ n. The details are described in Section 2.4. In the sequel, we
assume that we are given a problem in the form of (1.3).

1.2 Simplex Method

Before we describe our QP algorithm, let us briefly review the (revised)

simplex method for LP [20]. It operates on linear programs, given as
(LP) minimize cT x
subject to Ax = b (1.4)
x ≥ 0,
where the number of variables n is at least as large as the number of
equality constraints m.

Consider pairs of index sets B, N ⊆ [n] with |B| = m, |N | = n−m,

and B ∪˙ N = [n]. Each such pair induces a partition of the variables x
into xB and xN , a partition of c into cB and cN , and a partition of A
into AB and AN , where AB collects the columns of A with indices in B
(analogously for AN ). If A−1B exists, the m-vector xB of basic vari-
ables and the objective function value z are uniquely determined by the
(n−m)-vector xN of nonbasic variables, namely
xB = A−1 −1
B b − AB AN xN , (1.5)
z = cTB A−1
B b + (cTN − cTB A−1
B AN ) xN . (1.6)
Assigning nonnegative values to xN such that the implied values of xB
are nonnegative as well yields a feasible solution x∗ to (1.4). It is a basic
feasible solution if the nonbasic variables are zero. A basis B (in the
sense of LP) is the set of basic variables (identifying B and xB ), we will
refer to it as LP-basis to avoid any confusion with the notion of a basis
for quadratic programs (defined in Section 2.2).

In each iteration, the simplex method replaces a basic variable by a non-

basic variable such that the resulting basis with corresponding basic
18 Chapter 1. Linear and Quadratic Programming

feasible solution y ∗ satisfies cT y ∗ ≤ cT x∗ . An iteration – called pivot

step – consists of three parts: the pricing, which either finds a non-
basic variable to enter the basis or certifies that the current solution
is optimal; the ratio test, which either determines the basic variable
that leaves the basis or states that the problem is unbounded; and the
update, which computes the new basis B 0 and the corresponding basic
feasible solution.

Pricing. Consider the vector γ := cTN − cTB A−1B AN of reduced costs. The
current basic feasible solution is optimal if γ ≥ 0, because then xN ≥ 0
implies γ T xN ≥ 0, i.e. the objective function value z in (1.6) cannot be
improved by increasing one of the nonbasic variables. In case γ T xN 6≥ 0,
the pricing step returns some index j with γj < 0. The corresponding xj
is the entering variable.

Ratio Test. Increasing xj by t > 0 decreases the objective function

value by γj t. The variables become linear functions in t, i.e. x∗j (t) = t
and x∗B (t) = x∗B − t A−1
B Aj using (1.5). The task of the ratio test is
to find the largest value of t such that x∗ (t) stays feasible, equiva-
lently x∗B (t) ≥ 0. If such a value t0 exists, there is an index i ∈ B
with x∗i (t0 ) = 0 and xi is the leaving variable. Otherwise the problem
is unbounded.

Update. The basis B is replaced by the new basis B 0 := B ∪ {j}\{i}.

Furthermore, some representation of the basis matrix AB has to be
maintained, suitable to compute cTB A−1 −1
B and AB Aj efficiently. The
representation must also be easy to update, i.e. the update should be
substantially cheaper than computing the representation from scratch
when AB changes to AB 0 . As we will see in the implementation of the
QP solver, the inverse of the basis matrix is a suitable representation in
our scenario (see Part II, Section 6.3).

Further details and proofs can be found in the book of Chvátal [20].
Chapter 2

A Simplex-Like
Quadratic Programming
Algorithm

In this chapter, we describe our generalization of the simplex method

to QP, based on a preliminary version of Gärtner [38]. We point out
the main differences to the usual simplex method for LP (referred to as
LP simplex in the following) and describe the generalized pivot step in
detail. Furthermore, we show how inequality constraints can be handled
efficiently. For the presentation of our method, we make the following
nondegeneracy assumption.

Assumption 2.1 (nondegeneracy of quadratic program)

Given a quadratic program in standard form (1.3), we assume the fol-
lowing:

(i) The rows of A are linearly independent, i.e. A has full (row) rank.

(ii) The subsystem AG xG = b has only solutions for sets G ⊆ [n] with
|G| ≥ m.
20 Chapter 2. A Simplex-Like QP Algorithm

Later in this chapter, we describe how to deal with these degeneracies.

Section 2.5 gives the details.

2.1 Karush-Kuhn-Tucker Conditions

In the LP simplex, the linear objective function is kept in a way that

allows to decide directly if the current solution is optimal. In case
of QP things are more complicated, since the objective function is no
longer linear. We will use the Karush-Kuhn-Tucker (KKT) conditions
for convex programming [79, 11] to decide optimality. In the most gen-
eral setting, these conditions hold for convex programs, i.e. minimizing
a convex objective function subject to convex equality and inequality
constraints. The following two theorems are specialized versions, they
provide necessary and sufficient optimality criteria for quadratic pro-
grams [11, Section 3.4].

Theorem 2.2 (KKT conditions for QP)

A feasible solution x∗ ∈ Rn to the quadratic program
(QP) minimize cT x + xTD x
subject to A x = b
x ≥ 0,
is optimal if and only if there exists an m-vector λ and an n-vector µ ≥ 0
such that
cT + 2x∗ TD = −λTA + µT ,
µT x∗ = 0 .

Our QP algorithm described below will solve subproblems without non-

negativity constraints. The following version of the KKT conditions is
known as the method of Lagrange multipliers.

Theorem 2.3 (KKT conditions for unconstrained QP)

A feasible solution x∗ ∈ Rn to the unconstrained quadratic program
(UQP) minimize cT x + xTD x
subject to A x = b ,
2.2. Basic Solutions 21

is optimal if and only if there exists an m-vector λ such that

cT + 2x∗ TD = −λTA .

Note, if the rows of A are linearly independent, then λ and µ are unique
in Theorem 2.2. The same holds for λ in Theorem 2.3.

Corollary 2.4 Any vector x∗ > 0 satisfying A x = b is an optimal

solution to (QP) if and only if it is an optimal solution to (UQP).

Proof. For x∗ > 0 the second condition of Theorem 2.2 implies µT = 0.

Thus, the first condition of Theorem 2.2 and the (only) condition of
Theorem 2.3 are equivalent. 2

2.2 Basic Solutions

As the LP simplex, the QP simplex iterates through a sequence of basic

feasible solutions, always improving the objective function value. In-
terpreted geometrically, the set of feasible solutions is a polyhedron P.
In case of LP, each basic feasible solution corresponds to a vertex of P,
hence there is always an optimal solution which is a vertex of P. In case
of QP, this is not true in general, as the following trivial example shows:
Minimize x2 subject to −1 ≤ x ≤ 1. Here P is a polytope in 1-space
with vertices −1 and 1, but the unique optimal solution is assumed at
point 0, in the interior of P. Thus we have to consider more than only
the basic feasible solutions in the sense of LP to find the optimum of a
quadratic program.

A (QP) basic feasible solution is characterized by a subset B of the

variables (a QP-basis, defined below) and numerical values x∗B for the
variables in B; variables not in B will always have value zero. A variable
is called basic if it is in B, nonbasic otherwise. Unlike in the LP sim-
plex, the basis B may contain more than m variables, and in this case
their values are not uniquely determined by only the constraints of the
quadratic program. Instead, x∗B will be the optimal solution to an un-
constrained subproblem as follows.
22 Chapter 2. A Simplex-Like QP Algorithm

Definition 2.5 (QP-Basis)

A subset B of the variables of a quadratic program in standard form
defines a QP-basis if and only if

(i) the unconstrained subproblem

(UQP(B)) minimize cTB xB + xTB DB xB
(2.1)
subject to AB xB = b
has a unique optimal solution x∗B > 0, and
(ii) AB has full (row) rank, i.e. rank(AB ) = m,1

where cB , DB and AB are the entries of c, D and A relevant for the

variables in B, respectively.

Obviously, x∗ with x∗i = 0 for all nonbasic variables xi is a feasible

solution to the original problem (1.3). In case of LP (i.e. D = 0), the
definition of a QP-basis specializes to the usual notion of basic feasible
solutions (in the nondegenerate case).

In the sequel we identify each basic variable with the corresponding

index, i.e. B ⊆ [n] and i ∈ B if and only if xi is basic. The nonbasic
variables are collected in N := [n] \ B.

The following theorem gives an upper bound on the maximum size of

a QP-basis. The idea of the proof is due to Gärtner [38].

Theorem 2.6 Every QP-basis B satisfies

|B| ≤ m + rank(D).

Proof. Suppose there exists a basis B with corresponding optimal

solution x∗B and |B| > m+rank(D). We will show that x∗B is not the
unique solution to (UQP(B)), thus contradicting the definition of a
QP-basis.

By the KKT conditions (Theorem 2.3), there exists a vector λ such that

cTB + 2x∗B TDB = −λTAB ,

1 Note that AB has at least m columns by Assumption 2.1 (ii).
2.3. Pivot Step 23

if and only if x∗B is optimal for (UQP(B)). It follows that any y ≥ 0 in

the intersection of the affine spaces

U := { y | y TDB = x∗ TDB } , V := { y | AB y = b }

is also an optimal solution to (UQP(B)). Since U and V are solution

spaces of linear equation systems, their dimension is determined by the
difference of the number of variables and the rank of the respective
matrices D and A, see e.g. [32]. With k := |B| and r := rank(D), we
get

dim(U ∩ V ) = dim(U ) + dim(V ) − dim(U +V )

≥ (k−r) + (k−m) − k
= k − (m+r) .

Let W := { y | y ≥ 0 }, then the polytope P := U ∩ V ∩ W has dimen-

sion at least k−(m+r) and is contained in the set of optimal solutions
of (UQP(B)). The assumption k > m+r implies dim(P) > 0, thus x∗B
is not unique. 2

Again, if D = 0, we recover the bound for LP-bases. The theorem is

a crucial ingredient of our method. Well-known methods to general-
ize the simplex algorithm to QP integrate D explicitly into the con-
straint matrix [106]. This is appropriate for large sparse problems and
corresponding solvers, because it does not increase the number of non-
zero entries in the problem description. In our scenario, however, this
method is inappropriate, because it results in a problem with many vari-
ables and many constraints. The lemma limits the influence of D by
considering its rank rather than its size. As we see in the next chapter,
the rank of D is small in our applications.

2.3 Pivot Step

The process of going from one basic solution to the next is called a
pivot step. In the LP simplex, the pricing is the part of the pivot step
that decides whether the current solution is optimal, and which variable
enters the basis if an improvement is possible.
24 Chapter 2. A Simplex-Like QP Algorithm

2.3.1 Pricing

Testing whether a nonbasic variable xj can improve the solution by

entering the current QP-basis B is done as follows. Let B̂ := B ∪ {j}
and consider the subproblem

(QP(B̂)) minimize cTB̂ xB̂ + xTB̂ DB̂ xB̂

subject to AB̂ xB̂ = b (2.2)
xB̂ ≥ 0 .

By the KKT conditions for QP (Theorem 2.2), x∗B̂ (with x∗j = 0) is an

optimal solution to (QP(B̂)) if and only if there exist vectors λ and µ ≥ 0
such that

cTB̂ + 2x∗B̂ T DB̂ = −λT AB̂ + µ (2.3)

x∗B̂ T µ = 0. (2.4)

Since x∗B > 0, µB = 0 holds using (2.4). Extracting x∗j from (2.3) gives

cTB + 2x∗B T DB + 2x∗j DB,j

T
= −λT AB (2.3a)
cj + 2x∗B T DB,j + 2x∗j T
Dj,j = −λ Aj + µj , (2.3b)

where DB,j is the j-th column of DB . Equations (2.3a) together with

the constraints of (2.1) determine x∗B̂ (with x∗j = 0) and λ by the linear
equation system
! ! !
λ b 0 AB
MB = , MB := . (2.5)
x∗B −cB ATB 2DB

We call MB the basis matrix (of B). By the definition of a QP-basis,

x∗B is the unique optimal solution to (UQP(B)) and AB has full row
rank. Thus, also λ is unique and MB is regular, i.e. MB−1 exists.

Using the values of x∗B and λ, (2.3b) uniquely determines µj . In case

µj is negative, x∗B̂ (with x∗j = 0) cannot be an optimal solution to
(QP(B̂)) because µj < 0 contradicts µ ≥ 0 in Theorem 2.2. On the
other hand, if µj ≥ 0 holds for all nonbasic variables xj , we found vec-
tors λ and µ ≥ 0 fulfilling the KKT conditions for the original problem,
i.e. x∗ with x∗N = 0 is an optimal solution to (QP). Hence, the current
2.3. Pivot Step 25

solution can be improved by entering variable xj into the basis if and

only if µj < 0.

Different strategies for choosing the entering variable are described in

Part II, Section 6.4.

2.3.2 Ratio Test

The second part of the LP simplex’s pivot step is the ratio test. It
decides which variable has to leave the basis if another variable enters
it. Unless the problem is degenerate, there is a unique choice for the
leaving variable. In the QP simplex, it can happen that a variable enters
the basis, but there is no leaving variable, so that the basis gets larger.
This is the case if the objective function reaches a local minimum (while
the entering variable is increased), before some other (basic) variable
goes down to zero. Moreover, it can happen that even if some leaving
variable is found, the solution at that point is not basic. In this case,
the pivot step continues, and more variables may leave the basis, until
another basic solution is discovered. The QP ratio test consists of three
steps which are described below.

Step 1. Starting with a QP-basis B and an entering variable xj , we

want to find a new basis B 0 ⊆ B ∪ {j} with better objective function
value. Define B̂ := B ∪ {j}, then x∗B̂ (with x∗j = 0) is the optimal (and
unique) solution to

(UQPtj (B̂)) minimize cTB̂ xB̂ + xTB̂ DB̂ xB̂

subject to AB̂ xB̂ = b (2.6)
xj = t

for t = 0. Furthermore, (UQPtj (B̂)) has a unique optimal solution x∗B̂ (t)
for each value of t, which is determined by
! ! !
λ(t) b Aj
MB = −t (2.7)
x∗B (t) −cB 2DB,j

and x∗j (t) = t. This follows from the KKT conditions for (2.6) (Theo-
rem 2.3), the regularity of MB , and some elementary transformations.
26 Chapter 2. A Simplex-Like QP Algorithm

In the sequel, we use the following equivalent formulation

! ! ! ! !
λ(t) λ qλ qλ −1 Aj
= −t , := MB . (2.8)
x∗B (t) x∗B qx qx 2DB,j

While increasing t starting from zero (which improves the current solu-
tion), two things may happen. Either one of the basic variables becomes
zero, or a local minimum of the objective function is reached. The first
event can be tested by solving x∗i (t) = 0 for all i ∈ B. To check the
second event, we look at the linear function µj (t) derived from (2.3b)
and (2.8)

µj (t) = cj + ATj λ(t) + 2DB,j

T
x∗B (t) + 2Dj,j x∗j (t)
= µj + t (2Dj,j − ATj qλ − 2DB,j
T
qx ) . (2.9)

As soon as µj (t) becomes zero for some t > 0, λ(t) and µ = 0 fulfill the
KKT conditions of Theorem 2.2, thus x∗B̂ with x∗j (t) = t is an optimal
solution to (QP(B̂)). The following lemma shows that B̂ is the new
basis in case of the second event.

Lemma 2.7 If µj (t) in (2.9) becomes zero for some t > 0 before any
of the basic variables vanishes, then B̂ is the new basis.

Proof. We already know that x∗B̂ (t) is an optimal solution to (QP(B̂)),

if µj (t) = 0. We will show that MB̂ is regular, which directly implies
that x∗B̂ (t) is unique and that AB has full row rank.

Consider the matrices MB and MB̂ . The latter is just the basis
matrix MB plus one additional row and one additional column for
index j, which can be assumed to be the largest index in B, w.l.o.g.
It is easy to verify that
   
1 0 0 1 qλ
.. ..   ..   ..
 . .   MB .  . 
MB̂ = 
 qx 
 (2.10)
 1 0   0 
  1 
qλT qxT 1 0 ... 0 ν 0 ... 0 1

holds, where
ν := 2Dj,j − ATj qλ − 2DB,j
T
qx .
2.3. Pivot Step 27

The determinant of MB̂ is

det(MB̂ ) = det(MB ) ν ,

i.e. MB̂ is regular if ν 6= 0, since det(MB ) 6= 0 follows from the fact that
MB is a basis matrix. Suppose ν = 0, then by (2.9)

µj (t) = µj + t ν = µj < 0 ,

a contradiction to µj (t) = 0. 2

In case the first event happens, i.e. some basic variable xi becomes
zero, we implicitly add the constraint xi = 0 to (UQPtj (B̂)) by removing
index i from B. If MB\{i} is regular, we still have a unique optimal
solution to (UQPtj (B̂ \{i})) for each value of t and Step 1 is iterated.
Otherwise we proceed to Step 2.

Step 2. Let B be the set of basic variables after the last iteration
of Step 1. Since MB has become singular, (2.7) does no longer deter-
mine unique solutions to (UQPtj (B̂)) for arbitrary t (with B̂ = B ∪ {j}).
Reconsidering the KKT conditions for (QP(B̂)), i.e. Equations (2.3)
and (2.4), yields
     
λ b 0
x∗  −c 
MB̂  B  =  B  + µj  0  . (2.11)
 

x∗j −cj 1

In case MB̂ is singular, we proceed directly to Step 3. Otherwise, the

system of linear equations (2.11) has a unique solution for each value
of µj . The solutions are determined by a linear function in µj , which
can be written as
! ! !
λ(µj ) λ pλ
= + µj ,
x∗B̂ (µj ) x∗B̂ pxB̂

with    
pλ 0
p  −1 
:= M  .
0 (2.12)

 xB  B̂
pxj 1
28 Chapter 2. A Simplex-Like QP Algorithm

Any solution x∗B̂ (µj ) is feasible for (UQP(B̂)), and it is optimal if µj = 0.

Let t1 be the value of t for which the first event occured in the last
iteration of Step 1, then x∗B̂ (µj (t1 )) is the current feasible solution at
the beginning of Step 2.

While growing µj from µj (t1 ) towards zero (similar to t in Step 1), two
events may occur. Again, either one of the remaining basic variables
becomes zero, or a local minimum of the objective function is reached.
If the latter happens, i.e. µj equals zero, we found an optimal solu-
tion x∗B̂ (0) > 0 to (UQP(B̂)), which is at the same time an optimal
solution to the constrained problem (QP(B̂)) by Corollary 2.4. The
uniqueness of the solution follows from the regularity of MB̂ , which also
implies that B̂ is the new basis in that case.

On the other hand, if some basic variable xk becomes zero (first event),
we implicitly add the constraint xk = 0 to (UQP(B̂)) by removing k
from B̂. If MB̂\{k} stays regular, we still get unique solutions of (2.11)
for arbitrary values of µj . In this case Step 2 is iterated, otherwise we
continue with Step 3.

Step 3. Let B̂ be the set of basic variables (including xj ) after Step 2.

Then, by Lemma 2.8 below, the linear equation system (2.11) has only
solutions if µj = 0, hence any solution of (2.11) is already an optimal
solution to (UQP(B̂)). But, since MB̂ is singular, B̂ is not a QP-basis
yet. Using the polytope P from the proof of Theorem 2.6 (with B̂
instead of B), we find a basis B 0 B̂ with the same objective function
value as follows.

We initialize B 0 with B̂. Starting from x∗B̂ (which obviously belongs

to P), we decrease the value of some variable while maintaining the
invariant xB 0 ∈ P, until the first variable, say xi , becomes zero. Note
that xi can be the decreased variable itself or any of the other variables.
Fixing xi to zero, equivalently replacing P with the intersection of P
and the hyperplane { x | xi = 0 }, reduces the dimension of P by one.
We remove i from B 0 and iterate this step until the dimension of P
becomes zero. Then P consists of the single point x∗B 0 , which is the
unique solution to (UQP(B 0 )).
2.3. Pivot Step 29

Lemma 2.8 In case MB̂ is singular after Step 2, then the linear equa-
tion system (2.11) has only solutions if µj = 0.

Proof. The right-hand-side of (2.11) is a vector-valued linear function

in µj . Using Gaussian elimination, MB̂ can be made triangular while
the right-hand-side stays linear in µj (see e.g. [32, Section 0.4]). Thus,
any solution to (2.11) can be written as a linear polynomial in µj , i.e.
     
λ yλ zλ
x∗  y  z 
 B =:  B + µj  B .
x∗j yj zj
If µj 6= 0, the vector z has to fulfill
   
zλ 0
z   
MB̂  B  =  0  . (2.13)
zj 1
Showing that (2.13) has no solution proves the lemma.

There exists a vector r 6= 0 with rT MB̂ = 0 because MB̂ is singular.

Assuming that (2.13) has a solution z ∗ gives
 
0
rj = r  0  = rT MB̂ z ∗ = 0 .
T 
(2.14)


1
On the other hand, we get rj 6= 0 for all r 6= 0 with rT MB̂ = 0 by the
following case distinction, contradicting the assumption that (2.13) has
a solution.

If there has been no iteration in Step 2, i.e. both MB and MB̂ are
singular after Step 1, let xi with xi (t) = 0 be the last removed variable.
Then MB∪{i} is regular and (2.8) reads as
   
qλ Aj
MB∪{i}  qx  = 2DB,j  ,
   

qxi 2Di,j
which gives ! ! !
qλ Ai Aj
MB + qxi =
qx 2DB,i 2DB,j
30 Chapter 2. A Simplex-Like QP Algorithm

T
if the last row is ignored. Now, for all r0 6= 0 with r0 MB = 0 holds
!
T A j
r0 6= 0, (2.15)
2DB,j
T
since (ATi |2DB,i )r0 6= 0 follows from the regularity of MB∪{i} and qxi 6= 0
is implied by xi (t) = 0. Suppose there exists a vector r̃ 6= 0 with r̃j = 0
and r̃T MB̂ = 0. Then
 T !
T r̃B 
T

 T Aj 
r̃ MB̂ = MB̂ = r̃B MB  r̃B
0 2DB,j
shows !
T Aj
r̃B = 0,
2DB,j
contradicting (2.15), which completes the first case.

The second case occurs when there has been at least one iteration in
Step 2. Let xk with xk (µj ) = 0 be the last removed variable, then
MB̂∪{k} is regular and (2.12) gives
   
pλ 0
p   
 x  0
MB̂∪{k}  B  =   .
pxk   0 
pxj 1
By ignoring the ‘k’-th row, we obtain
     
pλ Ak 0
p  2D 
MB̂  xB  +  B,k  pxk =  0  .
 

pxj 2Dj,k 1
Finally, using the regularity of MB̂∪{k} and pxk 6= 0 (which is implied
by xk (µj ) = 0)), it follows that
   
0 Ak
rj = rT  0  = rT 2DB,k  zk 6= 0
   

1 2Dj,k
holds for all r 6= 0 with rT MB̂ = 0. This is a contradiction to (2.14)
and completes the proof. 2
2.4. Inequality Constraints 31

2.3.3 Update

The old basis B gets replaced by the new basis B 0 ⊆ B ∪ {j} which
was determined in the ratio test. Furthermore, some representation of
the basis matrix MB has to be maintained, which allows to compute
x∗B and λ of (2.5), qλ and qx of (2.8), and pλ and pxB̂ of (2.12) efficiently.
In addition, the update should be substantially cheaper than computing
the representation from scratch when MB changes to MB 0 . As already
suggested in the previous chapter, we will see that the inverse of the
basis matrix is a suitable representation in our scenario (cf. Part II,
Section 6.3).

2.4 Inequality Constraints

The efficiency of our method results from a special property of the

problems we consider. Usually, the number of constraints m, which is
a lower bound on the basis size (in the nondegenerate case), is very
small. Various problems even have m constant and in some cases we
are able to give an upper bound on the basis size that does not depend
on the possibly large number of variables n, (cf. Chapter 3). On the
other hand, we also want to solve problems with only few variables
but many (in)equality constraints, i.e. we have m  n. By introducing
slack variables in the usual way [20], we end up with a problem with
up to n+m variables and m constraints. Hence both numbers depend
on the possibly large m and we lost the special property of one very
small parameter of our original problem. The consequences are large
bases and a basis matrix of size at least (2m)×(2m). To overcome this
undesirable situation, we exploit the fact that only constraints holding
with equality determine the values of the original variables. These are
the equality constraints and those inequality constraints for which the
corresponding slack variable is nonbasic.

If the given quadratic program contains inequality constraints, they are

removed by introducing slack variables. Let E and S be the sets of
indices of the equality and inequality constraints, resp. For each i ∈ S,
we replace the inequality
32 Chapter 2. A Simplex-Like QP Algorithm

m
X
aij xj Q bi ,
j=1

by the equation
m
X
aij xj ± xsi = bi , (2.16)
j=1

where xsi is added, if the relation is ≤, and subtracted otherwise. The

matrix A is enlarged by |S| slack columns

Asi := ±ei , i ∈ S, (2.17)

where ei is the i-th unit vector, resulting in a quadratic program with

n+|S| variables and m equality constraints. Given a basis B, we want
to compute the current solution x∗B . By the KKT conditions (Theo-
rem 2.2), we have to solve the system

AB xB = b, (2.18)
ATB λ + 2DB xB = −cB . (2.19)

Define BO and BS to be the sets of indices of the original resp. slack

variables in the current basis, i.e. B = BO ∪˙ BS . Furthermore, parti-
tion S into two sets, such that SB and SN contain the indices of those
inequality constraints for which the corresponding slack variable is cur-
rently basic resp. nonbasic. Now, Equation (2.18) can be written as
   
AE ,BO AE ,BS ! bE
 xBO
 ASN,BO ASN,BS  =  bS N  ,
  
xBS
ASB,BO ASB,BS bS B

equivalently

AE ,BO xBO +AE ,BS xBS = bE , (2.20)

ASN,BO xBO +ASN,BS xBS = bSN , (2.21)
ASB,BO xBO +ASB,BS xBS = bSB . (2.22)

Two things follow from (2.17). Firstly, AE,BS and ASN,BS have only zero
entries, thus the second terms in (2.20) and (2.21) vanish. Secondly, the
values of the basic slack variables are determined by (2.22), i.e.

±xsi = bi − Ai,BO xBO , si ∈ B S ,

2.4. Inequality Constraints 33

where the sign of xsi is the same as in (2.16). Splitting up equa-

tion (2.19) in a similar manner yields
 
  λE
AT AT AT
 E,BO SN,BO SB,BO  λSN 
 
T T T
AE,BS ASN,BS ASB,BS  
λSB
    
2DBO,BO 2DBO,BS xBO −cBO
+  = ,
2DBS,BS 2DBS,BS xBS −cBS

respectively

ATE,BO λE + ATSN,BO λSN + ATSB,BO λSB

+ 2DBO,BO xBO + 2DBO,BS xBS = −cBO , (2.23)
ATE,BS λE + ATSN,BS λSN + ATSB,BS λSB
+ 2DBS,BO xBO + 2DBS,BS xBS = −cBS . (2.24)

Here, only DBO,BO has nonzero entries while all other parts of D have
just zero entries, and so has −cBS . Together with the known charater-
istics of A, equation (2.24) boils down to

±λi = 0, i ∈ SB .

Plugging this into (2.23) gives

ATE,BO λE + ATSN,BO λSN + 2DBO,BO xBO = −cBO .

Finally, we end up with the equation system

AE ∪S
˙ N,BO xBO = bE ∪S
˙ N (2.25)
ATE ∪S
˙ N,BO λE ∪S
˙ N + 2DBO,BO xBO = −cBO (2.26)

where the number of variables and the number of constraints is bounded

by
|E| + |SN | + |BO | ≤ min{n, m} + n.

Here, we estimated |E|+|SN | in two ways: |E|+|SN | ≤ |E|+|S| = m

and |E|+|SN | = |E|+|S|−|SB | = m−|BS | = m−|B|+|BO | ≤ n.
34 Chapter 2. A Simplex-Like QP Algorithm

Summarizing, we found a small representation of the equation system

given in (2.18) and (2.19). The size of the new representation is at
most (min{n, m}+n)×(min{n, m}+n), which is (2n)×(2n) if m > n
— a tremendous improvement for m  n over the original size of at
least (2m)×(2m). Solving the new equation system gives the values
of x∗BO and λE ∪S ∗ ∗
˙ N , the entries of xBS are computed directly from xBO ,
and λSB is the zero vector.

2.5 Degeneracies

According to Assumption 2.1 (i), we have to cope with linearly depen-

dent rows of A. We address the problem by either transforming Ax = b
into an equivalent system A0 x = b0 with rank(A0 ) = m or by showing
that there is no solution at all, meaning that the optimization problem
is infeasible.

To test the rank condition of a given equation system Ax = b, we bring

matrix A into triangular form using some standard method, e.g. Gaus-
sian elimination. Let A0 x0 = b0 be the resulting system with

a01,1 a01,2 ... a01,m ... a01,n

 
 0 a02,2 ... a02,m ... a02,n 
A0 =  , (2.27)
 
.. .. .. .. ..
 . . . . . 
0 ... 0 a0m,m ... a0m,n

where a0j,j 6= 0 holds if at least one nonzero entry exists in row j. This
can be always achieved by swapping the columns of A0 appropriately.2

In case we end up with all entries of the diagonal a01,1 to a0m,m being
nonzero, then A0 and A have full (row) rank m and we are done. Other-
wise some a0j,j = 0, i.e. row j of A0 has only zero entries, thus we found
a linearly dependent row of A. If the corresponding b0j is zero too, we
just remove constraint j from the equation system. This is done for
all ‘zero’ rows, resulting in an equivalent system where the constraint
2 The vector x0 contains the entries of x in possibly different order. A pair of

entries (xi , xj ) is swapped in x0 if the corresponding columns of A were swapped

in A0 during the transformation.
2.6. Conclusion 35

matrix has full row rank. Only if b0j 6= 0 holds, constraint j cannot be
fulfilled and the original equation system Ax = b has no solution.

If there exists a solution of Ax = b with less than m nonzero entries,

then we guarantee the second condition of Assumption 2.1 to hold using
symbolic perturbation. The right-hand-side of the equation system is
perturbed by the vector ε := (, 2 , . . . , m )T for some 0 <  < 1. Any
solution of the resulting equation system

Ax = b + ε (2.28)

is a vector-valued polynomial in , i.e.

  0
x1 + x11  + x21 2 + · · · + xm m
 
x1 () 1 
 x2 ()   x2 + x12  + x22 2 + · · · + xm
  0 m
2 
 

x= .
 .  
= . . (2.29)
 .   .. 

xn () x0n + x1n  + x2n 2 + · · · + xm
n
m

Choosing  appropriately ensures that no solution has less than m non-

zero entries. However, this choice has never to be done explicitly as
we will see in the implementation of the QP solver (cf. Part II, Sec-
tion 6.2.4).

2.6 Conclusion

We presented an algorithm for solving quadratic programming prob-

lems. The solver is a generalization of the simplex method to quadratic
programs. Unlike existing solvers, it is efficient if the problem is dense
and has either few variables or few constraints. Inequality constraints
are handled implicitly and degenerate problems are treated using sym-
bolic perturbation.
Chapter 3

Geometric Optimization
Problems

In this chapter we present four geometric optimization problems that

can be formulated as instances of LP or QP.

3.1 Polytope Distance

Consider two point sets P = { p1 , . . . , pr } and Q = { q1 , . . . , qs } with

r+s = n in d-dimensional space. The polytopePr distance problem Ps deals
with minimizing kp−qk such that p = i=1 λi pi and q = i=1 µi qi ,
see Figure 3.1 (primal version). The λi and µi are in [0, 1] and sum up
to one. This can be written as a quadratic program in n variables and
two constraints, namely
(PD) minimize xT C T C x
Pr
subject to xi = 1
Pi=1
n (3.1)
i=r+1 xi = 1
x ≥ 0,
where C = (p1 , . . . , pr ,−q1 , . . . ,−qs ). Here, D = C T C is an n×n-matrix,
but its rank is only d. Hence, by Theorem 2.6, the QP simplex will trace
38 Chapter 3. Geometric Optimization Problems

Figure 3.1: Polytope distance (primal and dual version)

only bases of size d+2 at most. This proves that the closest features of
two d-polytopes are always determined by at most d+2 points. (If the
polytopes have positive distance, d+1 points suffice.)

Note that the n×n-matrix D in (3.1) is dense. An equivalent formula-

tion of (PD) can be obtained by introducing a d-vector y of additional
variables with y = Cx, resulting in

(PD0 ) minimize yT y
subject to y = Cx
Pr
xi = 1 (3.2)
Pni=1
i=r+1 xi = 1
x ≥ 0.

This quadratic program has n+d variables and d+2 constraints, but
now the objective function has a sparse representation.

The dual version of the polytope distance problem is the following.

} and H = { h−
Given two sets of halfspaces G = { g1− , . . . , gr−T . . . , h−
1 ,T s }
r s
in dimension d with r+s = n, find points g ∈ i=1 gi and h ∈ j=1 h−
−
j
such that kg−hk is minimized, see Figure 3.1 (dual version). A point x is
contained in halfspace gi− if the function gi (x) defining the correspond-
ing hyperplane is non-positive (the same holds for h− j ). We obtain the
quadratic program
3.2. Smallest Enclosing Ball 39

(PD00 ) minimize (x−y)T (x−y)

subject to gi (x) ≤ 0 i = 1...r (3.3)
hj (y) ≤ 0 j = 1...s,
where the d-vectors x and y are the 2d variables. The n constraints are
linear, since the expressions gi (x) ≤ 0 and hj (y) ≤ 0 boil down to inner
product computations.

3.2 Smallest Enclosing Ball

Let P = { p1 , . . . , pn } be an n-point set. The smallest enclosing ball

problem is to find a point p∗ such that maxni=1 kpi −p∗ k is minimized.
The point p∗ is then the center of the smallest enclosing ball of P ,
see Figure 3.2. The following theorem shows that this problem can be
written as a quadratic program, thus generalizing a result of Rajan [83]
on the smallest enclosing sphere of d+1 points in Rd .

Theorem 3.1 Given an n-point set P = {p1 , . . . , pn }, define the d×n-

matrix C := (p1 , . . . , pn ), consider the quadratic program
Pn
(MB) minimize xT C T C x − i=1 pTi pi xi
Pn
subject to i=1 xi = 1 (3.4)
x ≥ 0,
and let (x∗1 , . . . , x∗n ) be some optimal solution. Then the point
n
X
p∗ = pi x∗i ,
i=1

is the center of the smallest enclosing ball of P and the negative value
of the objective function at x∗ is the squared radius of this ball.

Proof. By the Karush-Kuhn-Tucker conditions for convex quadratic

optimization problems (Theorem 2.2), there exists a 1-vector (i.e. a
number) λ and an n-vector µ ≥ 0, such that

2 x∗ T C T pk − pTk pk = −λ + µk k = 1...n, (3.5)

∗T
x µ = 0, (3.6)
40 Chapter 3. Geometric Optimization Problems

Figure 3.2: Smallest enclosing ball

Pn
if and only if x∗ is optimal for (MB). Using Cx∗ = i=1 pi x∗i gives

k pk −p∗ k2 = pTk pk − 2(Cx∗ )T pk + p∗ T p∗ = λ − µk + p∗ T p∗ .

Hence, p∗ is the center of a ball B containing P with squared radius

at most λ + p∗ T p∗ . Moreover, since either x∗k = 0 or µk = 0 for all
k ∈ {1, . . . , n} by (3.6), there exists some µj = 0 and the squared radius
of B is exactly r2 := λ + p∗ Tp∗ . Let S be the set of points pj with
positive coefficients, i.e. those points of P that lie on the boundary
of B (because x∗j > 0 implies µj = 0). Thus, p∗ ∈ conv(S) and B is the
smallest enclosing ball of P by Lemma 3.2.

Summing up (3.5) multiplied by x∗k gives

n
X n
X
2 x∗ T C T C x∗ − pTk pk x∗k = (−λ + µk ) x∗k = −λ.
k=1 k=1
Pn
The second equation holds because k=1 x∗k = 1 and either x∗k = 0 or
µk = 0. Subtracting p∗ T p∗ from both sides completes the proof. 2

The following lemma is well known. We use a proof of Seidel [93].

3.2. Smallest Enclosing Ball 41

Lemma 3.2 Let S be a set of points on the boundary of some ball B

with center c. Then B is the smallest enclosing ball of S, if and only if
c lies in the convex hull of S.

Proof. Let u be some unit vector and µu := minp∈S { (p−c)T u }. For

λ ≥ 0, define xu (λ) := c + λu and consider the smallest ball Bu (λ) con-
taining S with center xu (λ). For all p ∈ S,
kxu (λ)−pk2 = (c−p)T (c−p) + λ2 uT u − 2λ(c−p)T u
is maximized when the third term is minimized, i.e. if (c−p)T u = µu .
Thus, the squared radius of Bu (λ) is
ru (λ) := R2 − µ2u + (λ−µu )2 .

Observe, every point x can be uniquely represented in the form xu (λ)

with λ ≥ 0 (except for x = c, where the representation is not unique).
The squared radius ru (λ) is minimized for λ = µu if µu ≥ 0, and for λ = 0
otherwise. Furthermore, µu < 0 holds if and only if there exists no
hyperplane orthogonal to u which separates c and S.

If c lies in the convex hull of S then c is not separable from S. Thus,

µu < 0 implies λ = 0 and B = Bu (0) is the smallest enclosing ball of S.
On the other hand, if c ∈
/ conv(S), there exists a unit vector u orthogonal
to some hyperplane separating c and S. Here, Bu (µu ) contains S but
has smaller radius than B. 2

As before in the polytope distance problem, the matrix D = C T C has

rank at most d, so Theorem 2.6 shows that the optimal basis has size
at most d+1, proving that d+1 points suffice to determine the smallest
enclosing ball.

Again, D is a dense n×n-matrix. At the cost of adding d additional con-

straints and d additional variables, the following equivalent formulation
is obtained.
Pn
(MB0 ) minimize y T y − i=1 pTi pi xi
subject to y = Cx
Pn (3.7)
i=1 xi = 1
x ≥ 0.
42 Chapter 3. Geometric Optimization Problems

3.3 Smallest Enclosing Annulus

An annulus is the region between two concentric spheres. The small-

est enclosing annulus is the annulus of minimal difference between its
squared radii that covers a given point set, see Figure 3.3. In two dimen-
sions, this is the annulus of smallest area. Compared to the minimum-
width annulus [2] having minimal difference between its radii, our opti-
mization criterion can be seen as minimizing that difference scaled by
the ‘size’ of the annulus, namely by the sum of the two radii.

For an n-point set P = { p1 , . . . , pn } in d-space, let r and R denote the

small respectively large radius of the annulus covering P , and let c be
the annulus’ center. The objective is to minimize R2 − r2 subject to the
constraints
r ≤ kpi −ck ≤ R i = 1...n,

equivalently

r2 ≤ (pi1 −c1 )2 + · · · + (pid −cd )2 ≤ R2 i = 1...n,

using pi = (pi1 , . . . , pid ) and c = (c1 , . . . , cd ). Defining

α := r2 − (c21 + · · · + c2d ) , β := R2 − (c21 + · · · + c2d ) ,

yields a linear program in d+2 variables and 2n constraints,

(MA) minimize β−α

Pd
subject to j=1 2pij cj ≤ pi Tpi − α i = 1...n (3.8)
Pd i T
j=1 2pj cj ≥ pi pi − β i = 1...n,

where α, β, and c1 , . . . , cd are unknown. From an optimal solution

(α∗ , β ∗ , c∗1 , . . . , c∗d ) to (MA) the squared radii r2 and R2 can be computed
as
r2 = α∗ + k c∗ k2 , R2 = β ∗ + k c∗ k2 .

There is also a dual version of (3.8), namely the following linear program
in 2n variables λ := (λ1 , . . . , λn ) and µ := (µ1 , . . . , µn ) with d+2 con-
straints
3.4. Optimal Separating Hyperplane 43

Figure 3.3: Smallest enclosing annulus

Pn T Pn T
(MA’) minimize i=1pi pi µi − i=1 pi pi λi
Pn n
2pij λi + i=1 2pij µi = 0
P
subject to j = 1...d
Pni=1
λi = 1 (3.9)
Pi=1
n
i=1µi = 1
λ, µ ≥ 0 .

3.4 Optimal Separating Hyperplane

Given two point sets P and Q with a total of n points in d-dimensional

space, test whether they can be separated by a hyperplane, and if so,
find a separating hyperplane that maximizes the distance to the nearest
point, see Figure 3.4. The separation itself can be done by LP, but find-
ing the optimal hyperplane is QP. The problem can be reduced to the
primal version of the polytope distance problem. The optimal hyper-
plane is just the bisector of the segment connecting p and q (cf. Sec-
tion 3.1)
44 Chapter 3. Geometric Optimization Problems

Figure 3.4: Optimal separating hyperplane

3.5 Conclusion

We presented four geometric optimization problems that can be formu-

lated as quadratic programs (one of them even as a linear program).
All problems share the property of having min(n, m) small. For the
QP problems the rank of D is bounded by the dimension d of the ambi-
ent geometric space, which is usually small in the applications. Hence,
our QP simplex algorithm described in the previous chapter is suitable
to solve these problems.
Chapter 4

A Non-Quadratic
Geometric Optimization
Problem

A natural generalization of the smallest enclosing ball problem is that

of finding the smallest enclosing ellipsoid of a set of points. It is an
instance of convex programming and can be solved by general methods
in time O(n) if the dimension is fixed. The problem-specific parts of
these methods are encapsulated in primitive operations that deal with
subproblems of constant size. We derive explicit formulae for the primi-
tive operations of Welzl’s randomized method [105] in dimension d = 2.
The formulae contain only rational expressions, allowing for an exact
solution.

4.1 Smallest Enclosing Ellipsoid

Given a point c ∈ Rd and a positive definite matrix1 M ∈ Rd×d , the set

of points x ∈ Rd satisfying
1 i.e. xTM x > 0 holds for all x 6= 0.
46 Chapter 4. A Non-Quadratic Geom. Optimization Problem

Figure 4.1: Smallest enclosing ellipsoid

(x − c)T M (x − c) = 1 (4.1)

defines an ellipsoid with center c. The function f (x) = (x−c)T M (x−c)

is called the ellipsoid function, the set E = {x ∈ Rd | f (x) ≤ 1} is the
ellipsoid body. Note, varying a pair of entries symmetric to the main
diagonal of M in (4.1) does not change the ellipsoid as long as the sum
remains the same. In the sequel, we assume M to be symmetric, w.l.o.g.
The volume of the ellipsoid body E is

Vol(Sd )
Vol(E) = p , (4.2)
det(M )

where Sd is the d-dimensional unit sphere. This can be easily seen by

choosing the coordinate system according to the principal axes, such
that the defining matrix M becomes diagonal [88].

Given a point set P = {p1 , . . . , pn } ⊂ Rd , we are interested in the ellip-

soid body of smallest volume containing P . Identifying the body with
its generating ellipsoid, we call this the smallest enclosing ellipsoid of P ,
denoted by SmEll(P ), see Figure 4.1. (If P does not span Rd , then
SmEll(P ) is a lower-dimensional ellipsoid ‘living’ in the affine hull
of P ). The problem of finding SmEll(P ) can be written as the convex
program [76, Section 6.5]
4.2. Welzl’s Method 47

(ME) minimize − log det(M )

subject to (pi −c)T M (pi −c) ≤ 1 ∀ pi ∈ P (4.3)
M positive definite ,

where the entries of M and c are unknown. Since M is assumed to

be symmetric, (ME) has d(d+3)/2 variables and n constraints. The
objective function is quadratic only if d = 2, while the constraints are
convex sets. Thus, (ME) is not a quadratic program and we cannot use
our QP simplex method to solve it. Instead, we will use an algorithm
of Welzl [105] described in the next section.

4.2 Welzl’s Method

Let us briefly describe Welzl’s randomized algorithm for computing the

smallest enclosing ellipsoid of an n-point set in d-space [105]. The algo-
rithm is very simple and achieves an optimal expected runtime of O(n)
if d is constant.

We start with the following important facts (proofs of which may be

found in [88, 62]). By SmEll(Q, R) we denote the smallest ellipsoid
containing Q that has R on the boundary.

Proposition 4.1
(i) If there is any ellipsoid with R on its boundary that
encloses Q, then SmEll(Q, R) exists and is unique.

(ii) If SmEll(Q, R) exists and q ∈

/ SmEll(Q\{q}, R), then
SmEll(Q\{q}, R ∪ {q}) exists and equals SmEll(Q, R).

(iii) If SmEll(Q, R) exists, then there is a subset S ⊆ Q

with |S| ≤ max(0, d(d+3)/2−|R|) and
SmEll(Q, R) = SmEll(S, R) = SmEll(∅, S ∪ R).

By (iii), SmEll(P ) is always determined by a support set S ⊆ P with

at most δ := d(d+3)/2 support points. The value of δ coincides with
the number of free variables in the ellipsoid parameters M and c.
48 Chapter 4. A Non-Quadratic Geom. Optimization Problem

SmEll(P \{q}) SmEll(P )

Figure 4.2: The inductive step in Welzl’s algorithm

The idea of Welzl’s algorithm for computing SmEll(P ) is as follows:

if P is empty, SmEll(P ) is the empty set by definition. If not, choose a
point q ∈ P and recursively determine E := SmEll(P \{q}). If q ∈ E,
then E = SmEll(P ) and we are done. Otherwise, q must lie on the
boundary of SmEll(P ), and we get SmEll(P ) = SmEll(P \{q}, {q}),
see Figure 4.2. Computing the latter (in the same way) is now an
easier task because one degree of freedom has been eliminated. If the
point q to be removed for the recursive call is chosen uniformly at ran-
dom among the points in Q, we arrive at the following randomized
algorithm. To compute SmEll(P ), we call the procedure with the
pair (P, ∅).
Algorithm 4.2 (computes SmEll(Q, R), if it exists)
SmEll(Q, R):
IF Q = ∅ OR |R| = δ THEN
RETURN SmEll(∅, R)
ELSE
choose q ∈ Q uniformly at random
E := SmEll(Q\{q}, R)
IF q ∈ E THEN
RETURN E
ELSE
RETURN SmEll(Q\{q}, R ∪ {q})
END
END
4.2. Welzl’s Method 49

Termination of the procedure is immediate because the recursive calls

decrease the size of Q. Correctness follows from Proposition 4.1 and
the observation that the algorithm – when called with (P, ∅) – main-
tains the invariant ‘SmEll(Q, R) exists’. To justify the termination
criterion ‘|R| = δ’, we need the following lemma proving that in this
case only one ellipsoid E with R on the boundary exists, i.e. we must
have E = SmEll(∅, R) = SmEll(Q, R). This is remarkable, because
in general, an ellipsoid is not uniquely determined by any δ points on
the boundary (for example, consider δ−1 points on the boundary of a
(d−1)-dimensional ellipsoid E 0 and some additional point q; then there
are many d-dimensional ellipsoids through E and q).

Lemma 4.3 If R attains cardinality δ during a call to SmEll(P, ∅),

exactly one ellipsoid E with R on its boundary exists.

Proof. By expanding (4.1), we see that an ellipsoid is a special second

order surface of the form
{ p ∈ R d | pT M p + 2 p T m + w = 0 } ,
defined by δ+1 parameters M ∈ Rd×d (symmetric), m ∈ Rd , w ∈ R.

For a point set R ⊆ Rd let S(R) denote the set of (δ+1)-tuples of para-
meters that define second order surfaces through all points in R. It is
clear that S(R) is a vector space, and we define the degree of freedom
w.r.t. R to be dim(S(R)) − 1. Obviously, the degree of freedom is at
least δ − |R|, since any point in R introduces one linear relation between
the parameters.

Now we claim that during Algorithm 4.2, the degree of freedom w.r.t. R
is always exactly δ−|R|. This is clear for R = ∅. Moreover, if q is added
to R in the second recursive call of the algorithm, the degree of freedom
goes down, which proves the claim. To see this, assume on the contrary
that dim(S(R)) = dim(S(R ∪ {q})), hence S(R) = S(R ∪ {q}). Then
it follows that q already lies on any second order surface through R,
in particular on SmEll(Q\{q}, R). But then the second recursive call
would not have been made, a contradiction.

Now the claim of the lemma follows: if |R| = δ, the degree of freedom
is 0, i.e. S(R) has dimension 1. Since a second order surface is invariant
under scaling its parameters, this means that there is a unique second
order surface, in this case an ellipsoid, through R. 2
50 Chapter 4. A Non-Quadratic Geom. Optimization Problem

The primitive operations of Welzl’s method are the computation of

E = SmEll(∅, R) and the test q ∈ E. As we will see in the next section,
they can be combined into one single operation. Before, we describe a
heuristic for tuning the algorithm that has proven to be very efficient
in practice.

The move-to-front heuristic. There are point sets on which the

algorithm does not perform substantially better than expected; on such
point sets, the exponential behavior in δ = Θ(d2 ) leads to slow imple-
mentations already for small d. Although for d = 2 the actual run-
time is still tolerable for moderately large n, a dramatic improvement
(leading to a practically efficient solution for large n as well) is ob-
tained under the so-called move-to-front heuristic. This variant keeps
the points in an ordered list (initially random). In the first recursive
call, q is chosen to be the last point in the list (restricted to the cur-
rent subset of the points). If the subsequent in-ellipsoid test reveals
q 6∈ SmEll(Q\{q}, R), q is moved to the front of the list, after the
second recursive call to SmEll(Q\{q}, R ∪ {q}) has been completed.
Although the move-to-front heuristic does not eliminate the algorithm’s
exponential behavior in δ, it significantly reduces the number of primi-
tive operations to be called. See [105] for further details and computing
times.

4.3 Exact Primitives in the Plane

In the two-dimensional case, the constant-size problems of Algorithm 4.2

involve smallest enclosing ellipses defined by up to five support points,
where the difficult case arises when the ellipse is defined by four support
points. As we show below, even if the points have rational coordinates,
the ellipse will typically have not, so in order to stay with rational
expressions, an explicit evaluation of the ellipse has to be avoided.

For a given point set P , Welzl’s method computes a support set S of P ,

provided the following primitive operation is available.

Given R ⊆ P , 3 ≤ |R| ≤ 5, such that

SmEll(∅, R) exists, and a query point q ∈ P \R, decide
whether q lies inside SmEll(∅, R).
4.3. Exact Primitives in the Plane 51

This operation — we call it the in-ellipse test — can be reduced to

a sign evaluation of a certain derivative. This leads to an elegant and
efficient method whose computational primitives are in-ellipse tests over
rational ellipses and evaluations of derivatives at rational values.

Our method to deal with is based on the concept of conics.

4.3.1 Conics

A conic C in linear form is the set of points p = (x, y)T ∈ R2 satisfying

the quadratic equation
C(p) := rx2 + sy 2 + 2 txy + 2 ux + 2 vy + w = 0 , (4.4)
r, s, t, u, v, w being real parameters. C is invariant under scaling the
vector (r, s, t, u, v, w) by any nonzero factor. After setting
   
r t u
M := , m := , (4.5)
t s v
the conic assumes the form
C = { pT M p + 2 pT m + w = 0 } . (4.6)
If a point c ∈ R2 exists such that M c = −m, C is symmetric about c
and can be written in center form as
C = { (p−c)T M (p−c) − z = 0 } , (4.7)
where z = cT M c − w. If det(C) := det(M ) 6= 0, a center exists and is
unique. Conics with det(C) > 0 define ellipses.

By scaling with −1 if necessary, we can assume w.l.o.g. that C is nor-

malized , i.e. r ≥ 0. If E is a normalized ellipse, q lies inside E if and
only if E(q) ≤ 0.

Let C1 and C2 be two conics, then the linear combination

C := λ C1 + µ C2 , λ, µ ∈ R
is the conic given by C(p) = λC1 (p)+µC2 (p). If p belongs to both C1
and C2 , then p also belongs to C.

Now we are prepared to describe the in-ellipse test, for |R| = 3, 4, 5.

52 Chapter 4. A Non-Quadratic Geom. Optimization Problem

4.3.2 In-ellipse Test, |R| = 3

It is well-known [101, 80, 88] that SmEll(∅, {p1 , p2 , p3 }) is given in

center form (4.7) by
3 3
1X 1X
c= pi , M −1 = (pi −c)(pi −c)T , z = 2.
3 i=1 3 i=1

From this, M is easy to compute. Query point q lies inside SmEll(∅, R)

if and only if (q−c)T M (q−c) − z ≤ 0.

4.3.3 In-ellipse Test, |R| = 4

SmEll(∅, R) is some conic through R = { p1 , p2 , p3 , p4 } (R being in con-

vex position). Any such conic is a linear combination of two special
conics C1 and C2 through R, see Figure 4.3 [98]. To see that these
are indeed conics, consider three points q1 = (x1 , y1 ), q2 = (x2 , y2 ), and
q3 = (x3 , y3 ) and define
 
x1 −x3 x2 −x3
[q1 q2 q3 ] := det . (4.8)
y1 −y3 y2 −y3

[q1 q2 q3 ] records the orientation of the point triple; in particular, if the

points are collinear, then [q1 q2 q3 ] = 0. This implies

C1 (p) = [p1 p2 p][p3 p4 p] , C2 (p) = [p2 p3 p][p4 p1 p] ,

and these turn out to be quadratic expressions as required in the conic

equation (4.4), easily computable from the points in R.

Now, given the query point q ∈ / R, there exists a unique conic C0 through
the five points R ∪ {q}, since R is an (intermediate) support set of Algo-
rithm 4.2, see also [98]. We can compute this conic as C0 = λ0 C1 +µ0 C2 ,
with λ0 := C2 (q) and µ0 := −C1 (q). In the sequel we assume that C0 is
normalized. Depending on the type of C0 we distinguish two cases.

Case 1. C0 is not an ellipse, i.e. det(C0 ) ≤ 0. Then we have the fol-

lowing result.
4.3. Exact Primitives in the Plane 53

C2

p4
C1 p3

p1
p2

Figure 4.3: Two special conics through four points

p4 p3

p1

p2

Figure 4.4: The two parabolas through four points

Lemma 4.4 Exactly one of the following statements holds.

(i) q lies inside any ellipse through R.

(ii) q lies outside any ellipse through R.

Let us give some intuition, before we formally prove the lemma. Since
no three of the four support points are collinear, there exist two (possi-
bly degenerate) parabolas through these points, see Figure 4.4. These
parabolas cut the plane into regions which determine the type of C0 .
Only if q lies strictly inside one parabola and strictly outside the other,
C0 is an ellipse. Otherwise, q either lies inside both parabolas in which
54 Chapter 4. A Non-Quadratic Geom. Optimization Problem

case q also lies inside all ellipses through p1 , p2 , p3 , and p4 , or q lies

outside both parabolas, also being outside all the ellipses.

Proof. Assume there are two ellipses E and E 0 through R, with E(q) ≤ 0
and E 0 (q) > 0. Then we find λ ∈ [0, 1) such that E 00 := (1−λ)E +λE 0
satisfies E 00 (q) = 0, i.e. E 00 goes through R ∪ {q}. Thus E 00 equals C0 and
is not an ellipse. On the other hand, the convex combination of two
ellipses is an ellipse again [88, Chapter 1], a contradiction. 2

Lemma 4.4 shows that it suffices to test q against any ellipse through
the four points to obtain the desired result. Let

α := r1 s1 − t21 , β := r1 s2 + r2 s1 − 2 t1 t2 , γ := r2 s2 − t22 ,

where ri , si , ti are the parameters of Ci in the linear form (4.4). Then

E := λC1 +µC2 with λ := 2γ −β and µ := 2α−β defines such an ellipse.
For this, one observes that

det(E) = (4αγ − β 2 )(α + γ − β).

We will show that both factors have negative sign, thus proving that
the choice of λ and µ indeed yields an ellipse E.

With definition (4.8) we can check that

4αγ − β 2 = −[p1 p2 p3 ][p2 p3 p4 ][p3 p4 p1 ][p4 p1 p2 ].

After a preprocessing, one can assume that p1 , p2 , p3 , p4 are in coun-

terclockwise order (they must be in convex position, because otherwise
SmEll(∅, {p1 , p2 , p3 , p4 }) does not exist). This means, each bracketed
term has positive sign, and 4αγ −β 2 < 0 follows. Moreover, we can
easily verify that

α + γ − β = [p2 p4 p1 ][p2 p4 p3 ] − (κ1 +κ2 )2 ,

where

κ1 := ((x1 −x2 )(y3 −y4 ) − (y1 −y2 )(x3 −x4 ))/2

κ2 := ((x2 −x3 )(y4 −y1 ) − (y2 −y3 )(x4 −x1 ))/2

with pi = (xi , yi ), i = 1 . . . 4. The product [p2 p4 p1 ][p2 p4 p3 ] is negative

because p1 and p3 lie on different sides of the diagonal p2 p4 . It follows
that α+γ −β < 0 and finally det(E) > 0.
4.3. Exact Primitives in the Plane 55

Case 2. C0 is an ellipse E, i.e. det(C0 ) > 0. We need to check the

position of q relative to E ∗ := SmEll(∅, R), given by

E ∗ = λ∗ C1 + µ∗ C2 ,

with unknown parameters λ∗ and µ∗ . In the form of (4.4), E is deter-

mined by r0 , . . . , w0 , where r0 = λ0 r1 +µ0 r2 . By scaling the representa-
tion of E ∗ accordingly, we can also assume that r0 = λ∗ r1 +µ∗ r2 holds.
In other words, E ∗ is obtained from E by varying λ and µ along the line
{λr1 +µr2 = r0 }. This means,
 ∗    
λ λ0 ∗ −r2
= + τ (4.9)
µ∗ µ0 r1

for some τ ∗ ∈ R. Define

E τ := (λ0 −τ r2 ) C1 + (µ0 +τ r1 ) C2 , τ ∈ R.
∗
Then E 0 = E and E τ = E ∗ . The function g(τ ) := E τ (q) is linear, hence
we get 
∗ ∗ ∂ τ
= ρ τ ∗,

E (q) = τ E (q)
∂τ τ =0

where ρ := C2 (q)r1 −C1 (q)r2 . Consequently, q lies inside SmEll(∅, R) if

and only if ρ τ ∗ ≤ 0.

The following lemma is proved in [22], see also [88].

Lemma 4.5 Consider two ellipses E1 and E2 , and let

E λ := (1−λ)E1 + λE2

be a convex combination, λ ∈ (0, 1). Then E λ is an ellipse satisfying

Vol(E λ ) < max(Vol(E1 ), Vol(E2 )).

Since E τ is a convex combination of E and E ∗ for τ ranging between 0

and τ ∗ , the volume of E τ decreases as τ goes from 0 to τ ∗ , hence
  
∗ ∂ τ 

sgn(τ ) = −sgn Vol(E ) .
∂τ τ =0
56 Chapter 4. A Non-Quadratic Geom. Optimization Problem

If E τ is given in center form (4.7), its area is

π
Vol(E τ ) = p ,
det(M/z)

following from (4.2). Consequently,

     
∂ τ 
 ∂ 
sgn Vol(E ) = −sgn det(M/z)
 .
∂τ τ =0 ∂τ τ =0

Recall that if M and m collect the parameters of E τ as in (4.5) with

c = M −1 m being its center, we get z = cTM c − w = mTM −1 m − w,
where M , m, and w depend on τ (which we omit in the sequel, for
the sake of readability). Noting that
 
1 s −t
M −1 = ,
det(M ) −t r
we get
1
z= (u2 s − 2 uvt + v 2 r) − w .
det(M )

Let us introduce the following abbreviations.

d := det(M ), Z := u2 s − 2 uvt + v 2 r .

With primes (d0 , Z 0 , etc.) we denote derivatives w.r.t. τ . Now we can

write  0
∂ d d0 z − 2 dz 0
det(M/z) = 2
= . (4.10)
∂τ z z3

Since d(0) and z(0) are positive (recall that E is a normalized ellipse),
this is equal in sign to

σ := d (d0 z − 2 dz 0 ) ,

at least when evaluated for τ = 0, which is the value we are interested in.
Furthermore, we have
1 d0
d0 z = d0 ( Z − w) = Z − d0 w ,
d d
0 0
Z d − Zd Z 0 d − Zd0
dz 0 = d ( − w 0
) = − dw0 ,
d2 d
4.3. Exact Primitives in the Plane 57

hence

σ = d0 Z − dd0 w − 2 (Z 0 d−Zd0 −d2 w0 ) = 3 d0 Z + d (2 dw0 −d0 w−2 Z 0 ) .

Rewriting Z as u (us−vt) + v (vr−ut) =: uZ1 + vZ2 , we get

d = rs − t2 , Z 0 = u0 Z1 + uZ10 + v 0 Z2 + vZ20 ,
d0 = r0 s + rs0 − 2 t t0 , Z10 = u0 s + us0 − v 0 t − vt0 ,
Z20 = v 0 r + vr0 − u0 t − ut0 .

For τ = 0, all these values can be computed directly from r(0), . . . , w(0)
(the defining values of E) and their corresponding derived values
r0 (0), . . . , w0 (0). For the latter we get r0 (0) = 0, s0 (0) = r1 s2 −r2 s1 , . . . ,
w0 (0) = r1 w2 −r2 w1 . We obtain that q lies inside SmEll(∅, R) if and
only if ρ σ(0) ≤ 0.

Note that for deciding the in-ellipse test in the case |R| = 4, it was not
necessary to know SmEll(∅, R) explicitly. In fact, SmEll(∅, R) is not
even representable with rational coordinates in general, as shown by the
following example.

Consider the points p1 = (0, 0), p2 = (1, 0), p3 = (1/2, 1), and p4 = (0, 1),
see Figure 4.5. As noted before, SmEll(∅, {p1 , p2 , p3 , p4 }) is a linear
combination λC1 +µC2 of two special conics C1 and C2 through the four
points, as depicted in Figure 4.3. By explicitly computing these conics,
one finds that the linear combination in the form of (4.6) is given by
   
µ µ/4 −µ/2
M= , m= , w = 0.
µ/4 −λ/2 λ/4

Standard calculus shows that this defines the ellipse √ of minimum vol-
ume through p1 , p2 , p3 , p4 if and only if 4λ = −(3+ 13)µ holds. This
means, the linear form of SmEll(∅, {p1 , p2 , p3 , p4 }) contains irrational
coordinates, no matter how it is scaled. This also holds for the center
form. In particular, the center c = (xc , yc ) evaluates to
√ √
9 + 3 13 1 + 13
xc = √ ≈ 0.406 , yc = √ ≈ 0.377 .
20 + 8 13 5 + 2 13
58 Chapter 4. A Non-Quadratic Geom. Optimization Problem

p4 p3

p1 p2

Figure 4.5: Irrational smallest ellipse through four points

4.3.4 In-ellipse Test, |R| = 5

It is easy to see that in Welzl’s method, R attains cardinality five only

if before, a test ‘p ∈ SmEll(∅, R\{p}) ?’ has been performed (with a
negative result), for some p ∈ R. In the process of doing this test, the
unique conic (which we know is an ellipse E) through the points in R has
already been computed, see previous subsection. Now we just ‘recycle’ E
to conclude that q lies inside SmEll(∅, R) if and only if E(q) ≤ 0. s

4.4 Conclusion

We have described primitives for Welzl’s method leading to a rational

arithmetic solution for the problem of computing the smallest enclosing
ellipse of a planar point set.

The output of the algorithm is a support set S. In addition, for |S| =

6 4,
our method determines SmEll(P ) = SmEll(S) = SmEll(∅, S) explic-
itly. For |S| = 4, the value τ ∗ defining SmEll(∅, S) via (4.9) appears
among the roots of (4.10); a careful analysis [80, 88] reduces this to a
cubic polynomial in τ , thus an exact symbolic representation or a float-
ing point approximation of τ ∗ and SmEll(∅, S) can be computed in a
postprocessing step.
4.4. Conclusion 59

Note that even a number type supporting k-th roots with arbitrary pre-
cision like leda real [68] can not handle the smallest enclosing ellipse
with four boundary points exactly, because third roots of complex num-
bers are needed to solve (4.10).

From a practical point of view, the three-dimensional version of the

problem is probably most interesting, and one might ask how our tech-
niques apply to this case. Welzl’s method as described in Section 4.2
works in any dimension, but the primitive operations are already not
sufficiently understood for d = 3. First of all, the number of basic
cases is larger; we need to do in-ellipsoid tests over ellipsoids defined by
4 ≤ k ≤ 9 boundary points. While the extreme cases k = 4 and k = 9
are easy (they behave similarly to the extreme cases k = 3, 5 for d = 2),
no exact method for any other case is known. Our ideas readily gen-
eralize to the case k = 8: here we can (as in the planar case) use the
fact that eight points – if they appear as a set R during Algorithm 4.2
– determine an ellipsoid up to one degree of freedom, see the proof of
Lemma 4.3. Beyond that, it is not clear whether the method generalizes.

In any dimension larger than two, an open problem is to prove the

existence of a rational expression whose sign tells whether a point q ∈ Rd
lies inside the smallest ellipsoid determined by d+1 ≤ k ≤ d(d+3)/2
boundary points. If such an expression exists, how can it be computed,
and what is its complexity?

An implementation of Welzl’s method in combination with our primi-

tives is available in the Computational Geometry Algorithms Library
described in the next chapter.
 Part II

Implementation
Chapter 5

Cgal, the
Computational
Geometry Algorithms
Library
The birth of Cgal dates back to a meeting in Utrecht in January 1995.
Shortly afterwards, the five authors of [31] started developing the kernel.
The development of the whole library has been made possible through
the funding of two projects1 by the European Community. Since the
official start of the Cgal project in October 1996, the team of devel-
opers has grown considerably. It consists mostly of research assistants,
PhD students and postdocs in academia, who are professionals in the
field of computational geometry and related areas. They form a hetero-
geneous team of developers; some of them working part time for Cgal,
some of them full time. The Cgal release 2.3 (August 2001) consists
of approximately 200,000 lines of C++ source code2 for the library, plus
100,000 lines for accompanying sources, such as the test suite and exam-
ple programs. Cgal’s WWW home-page3 http://www.cgal.org/ pro-
1 ESPRIT IV LTR Projects No. 21957 (CGAL) and No. 28155 (GALIA)
2 C++ comments and empty lines are not counted.
3 The reservation of Cgal’s own domain was proposed by the author during the

1999 Cgal Implementation Meeting at Schloß Dagstuhl.

64 Chapter 5. Computational Geometry Algorithms Library

vides a list of publications about Cgal and related research: previ-

ous overviews [78], the first design of the geometric kernel [30], recent
overviews and descriptions of the current design [31, 53].

5.1 Related Work

An overview on the state of the art of computational geometry software

before Cgal including many references is given in [3]. Three approaches
of implementing geometric software can be distinguished: collections
of standalone implementations, integrated applications, and software
libraries.

The approach of collecting isolated or only loosely coupled implementa-

tions usually requires some adaptation effort to use and combine such
algorithms. Although comparable to the collection of algorithms in the
successful Graphics Gems Series [49, 4, 59, 52], the adaptation of com-
putational geometry implementations is harder due to the need of more
involved data structures and more advanced algorithms. A good collec-
tion provides the Directory of Computational Geometry Software 4 .

Advantages of integrated applications and workbenches are homoge-

neous environments with animation and interaction capabilities. Dis-
advantages are monolithic structures which make them hard to extend
and hard to reuse in other projects. First implementation efforts were
started at the end of the Eighties [29, 24], in particular XYZ GeoBench5
[77, 92] developed at ETH Zurich is one of Cgal’s precursors.

If well designed, the components of a library work seamlessly together.

They can be reused in other projects and the library is extensible.
Examples are the precursors of Cgal developed by members of the
Cgal consortium: Plageo and Spageo [48], developed at Utrecht
University, C++gal [6], developed at Inria Sophia-Antipolis, and the
geometric part of Leda6 [67, 68], developed at Max-Planck-Institut für
Informatik, Saarbrücken. Another example is GeomLib [7], a compu-
tational geometry library implemented in Java at the Center for Geo-
4 http://www.geom.umn.edu/software/cglist/
5 http://wwwjn.inf.ethz.ch/geobench/XYZGeoBench.html
6 http://www.mpi-sb.mpg.de/LEDA/
5.2. Generic Programming 65

metric Computing, located at Brown University, Duke University, and

John Hopkins University in the United States. They state their goal as
an effective technology transfer from computational geometry to relevant
applied fields.

5.2 Generic Programming

Generic and flexible designs can be achieved following basically one of

the two paradigms; object-oriented programming or generic program-
ming. Both paradigms are supported in C++: Object-oriented pro-
gramming, using inheritance from base classes with virtual member
functions, and generic programming, using class templates and func-
tion templates. Both paradigms are also available in other languages,
but we stay with the notion used in C++, which is the choice made for
Cgal.

The flexibility in the object-oriented programming paradigm is achieved

with a base class, which defines an interface, and derived classes that
implement this interface. Generic functionality can be programmed in
terms of the base class and a user can select any of the derived classes
wherever the base class is required. The classes actually used can be
selected at runtime and the generic functionality can be implemented
without knowing all derived classes beforehand. In C++ so-called virtual
member functions and runtime type information support this paradigm.
The base class is usually a pure virtual base class.

The advantages are the explicit definition of the interface and the run-
time flexibility. But there are four main disadvantages: Firstly, the
object-oriented programming paradigm cannot provide strong type
checking at compile time whenever dynamic casts are used, which is
necessary in C++ to achieve flexibility. Secondly, this paradigm enforces
tight coupling through the inheritance relationship [61], thirdly, it re-
quires additional memory for each object (in C++ the so-called virtual
function table pointer ) and, fourthly, it adds for each call to a virtual
member function an indirection through the virtual function table [63].
The latter is of particular interest when considering runtime perfor-
mance since virtual member functions can usually not be made inline
and are therefore not subject to code optimization within the calling
66 Chapter 5. Computational Geometry Algorithms Library

function.7 Modern microprocessor architectures can optimize at run-

time, but, besides the difficulty of runtime predictions, these mecha-
nisms are more likely to fail for virtual member functions. These effects
are negligible for larger functions, but small functions will suffer a loss
in runtime of one or two orders of magnitude. Significant examples
are the access of point coordinates and arithmetic for low-dimensional
geometric objects (see for example [85]).

The generic programming paradigm features what is known in C++ as

class templates and function templates. Templates are incompletely
specified components in which a few types are left open and represented
by formal placeholders, the template arguments. The compiler gener-
ates a separate translation of the component with actual types replacing
the formal placeholders wherever this template is used. This process is
called template instantiation. The actual types for a function template
are implicitly given by the types of the function arguments at instanti-
ation time. An example is a swap function that exchanges the value of
two variables of arbitrary types. The actual types for a class template
are explicitly provided by the programmer. An example is a generic list
class for arbitrary item types. The following definitions would enable us
to use list<int>, with the actual type int given explicitly, for a list of
integers and to swap two integer variables x and y with the expression
swap(x,y), where the actual type int is given implicitly.

template < class T > class list {

// placeholder T represents the item type symbolically
void push_back( const T& t); // append t to the list
};

template < class T > void swap( T& a, T& b) {

T tmp = a; a = b; b = tmp;
}

The example of the swap function illustrates that a template usually

requires certain properties of the template arguments, in this case that
7 There are notable exceptions where the compiler can deduce for a virtual member

function the actual member function that is called, which allows the compiler to
optimize this call. The keyword final has been introduced in Java to support this
intention. However, these techniques are not realized in C++ compilers so far and
they cannot succeed in all cases, even though it is arguable that typical uses in
Cgal can be optimized. However, distributing a software library in precompiled
components will hinder their optimization, which must be done at link time.
5.2. Generic Programming 67

variables of type T are assignable. An actual type used in the template

instantiation must comply with these assumptions in order to obtain
a correct template instantiation. We can distinguish between syntactic
requirements (in our example the assignment operator is needed) and
semantic requirements (the assignment operator should actually copy
the value). If the syntactic requirements are not fulfilled, compilation
simply fails. Semantic requirements cannot be checked at compile time.
However, it might be useful to connect a specific semantic requirement
to an artificial, newly introduced syntactic requirement, e.g. a tag sim-
ilar to iterator tags in [99]. This technique allows decisions at compile
time based on the actual type of these tags.

The set of requirements that is needed to obtain a correct instantiation

of a member function of a class template is usually only a fraction
of all requirements for the template arguments of this class template.
If only a subset of the member functions is used in an instantiation of
a class template, it would be sufficient for the actual types to fulfill
only the requirements needed for this subset of member functions. This
is possible in C++, since as long as a C++ compiler is not explicitly
forced, the compiler is not allowed to instantiate member functions that
are not used, and therefore possible compilation errors due to missing
functionality of the actual types cannot occur [18]. This enables us to
design class templates with optional functionality, which can be used
if and only if the actual types used in the template instantiation fulfill
the additional requirements.

A good and well known example illustrating generic programming is

the Standard Template Library (Stl) [99, 18, 72, 96]. Generality and
flexibility have been achieved with the carefully chosen set of concepts,
where a concept is a well defined set of requirements. In our swap-
function example, the appropriate concept is named ‘assignable’ and
includes the requirement of an assignment operator [96]. If an actual
type fulfills the requirements of a concept, it is a model for this concept.
Here, int is a model of the concept ‘assignable’.

Algorithmic abstraction is a key goal in generic programming [73, 74].

One aspect is to reduce the interface to the data types used in the algo-
rithm to a set of simple and general concepts. One of them is the iterator
concept in Stl which is an abstraction of pointers. Iterators serve two
purposes: they refer to an item and they traverse over the sequence
of items that are stored in a data structure, also known as container
68 Chapter 5. Computational Geometry Algorithms Library

class in Stl. Five different categories are defined for iterators: input,
output, forward, bidirectional and random-access iterators, according
to the different possibilities of accessing items in a container class. The
usual C-pointer referring to a C-array is a model for a random-access
iterator.

A sequence of items is specified by a range [first,beyond) of two

iterators. This notion of a half-open interval denotes the sequence of
all iterators obtained by starting with first and advancing first until
beyond is reached, but it does not include beyond. A container class is
supposed to provide a local type, which is a model of an iterator, and two
member functions: begin() returns the start iterator of the sequence
and end() returns the iterator referring to the ‘past-the-end’-position
of the sequence.

Generic algorithms are not written for a particular container class in

Stl, they use iterators instead. For example, a generic contains func-
tion can be written to work for any model of an input iterator. It returns
true if and only if the value is contained in the values of the range
[first,beyond).

template < class InputIterator, class T >

bool contains( InputIterator first,
InputIterator beyond, const T& value) {
while ( ( first != beyond) && ( *first != value)) ++first;
return ( first != beyond);
}

The advantages of the generic programming paradigm are strong type

checking at compile time during the template instantiation, no need for
extra storage nor additional indirections during function calls, and full
support of inline member functions and code optimization at compile
time [100]. One specific disadvantage of generic programming in C++ is
the lack of a notation in C++ to declare the syntactical requirements for
a template argument, i.e. the equivalent of the virtual base class in the
object-oriented programming paradigm. The syntactical requirements
are scattered throughout the implementation of the template. The con-
cise collection of the requirements is left for the code documentation.
In general, the flexibility is resolved at compile time, which gives the
advantages mentioned above, but it can be seen as a disadvantage if
runtime flexibility is needed. However, the generic data structures and
5.3. Library Structure 69

algorithms can be parameterized with the base class used in the object-
oriented programming to achieve runtime flexibility where needed.

5.3 Library Structure

Cgal is structured into three layers and a support library, which stands
apart. The three layers are the core library with basic non-geometric
functionality, the geometric kernel with basic geometric objects and
operations, and the basic library with geometric algorithms and geo-
metric data structures.

The three layers and the support library are further subdivided into
smaller modular units, see Figure 5.1. The modular approach has sev-
eral benefits. The library is easier to learn, because it is possible for
a user to understand a small part without having any knowledge of
other parts. For building the library, the modules are a good way of
distributing implementation work among the project partners. Test-
ing and maintainance of the library is easier when there are only few
dependencies between units [61].

The geometric kernel contains simple geometric objects of constant

size. Examples are points, lines, planes, segments, triangles, tetrahe-
dra, circles, spheres and more. There are geometric predicates on those
objects, operations to compute intersections of and distances between
objects, and affine transformations. The geometric kernel is split up into
three parts, which deal with two-dimensional objects, three-dimensional
objects, and general-dimensional objects. Geometry in two and three
dimensions is well studied and has lots of applications, which is the rea-
son for the special status of the corresponding parts. For all dimensions
there are Cartesian and homogeneous representations available for the
coordinates.

To solve robustness problems, Cgal advocates the use of exact arith-

metic instead of floating point arithmetic. An arithmetic is associated
with a number type in Cgal and the classes in the geometric ker-
nel are parameterized by number types. Cgal provides own number
types [17, 16] and supports number types from other sources, e.g. from
Leda or the Gnu Multiple Precision Library [51]. Since the arithmetic
70 Chapter 5. Computational Geometry Algorithms Library

Basic Support
planar convex triangu−
Library polygon ... Library
map hull lation

visuali−
Geometric zation
two− three− d−
Kernel dimensional dimensional dimensional
number
types
Core
Library configuration assertions circulators ... ...

Figure 5.1: The structure of Cgal

operations needed in Cgal are quite basic, every library supplying num-
ber types can be easily adapted to work with Cgal.

The basic library contains more complex geometric objects and data
structures: polygons, planar maps, polyhedra and so on. It also contains
algorithms, such as computing the convex hull of a set of points, the
union of two polygons, smallest enclosing ellipse and so on. Figure 5.1
indicates the major parts in the basic library. These parts are mostly
independent from each other and even independent from the kernel.
This independence has been achieved with geometric traits classes as
described in Section 5.5.3 below.

The core library offers basic non-geometric functionality needed by the

geometric kernel or the basic library. Very important is the support for
different C++ compilers which all have their own limitations. The core
library also contains circulators and random number generators as well
as code for checking assertions, preconditions, and postconditions.

In contrast to the core library, the support library provides functionality

that the rest of the library does not depend on. Visualization and
external file formats are important aspects of the support library. The
list of supported formats contains Vrml and PostScript as well as the
GeomView program and Leda windows for 2D and 3D visualization.
The adaptation of number types from other libraries is contained in the
support library, too. The separation from the rest of the library makes
the functionality of the support library orthogonal and open for future
extensions.
5.4. Geometric Kernel 71

5.4 Geometric Kernel

The geometric kernel contains objects of constant size, such as point,

vector, direction, line, ray, segment, triangle, iso-oriented rectangle and
tetrahedron. Each type provides a set of member functions, for exam-
ple access to the defining objects, the bounding box of the object if
existing, and affine transformation. Global functions are available for
the detection and computation of intersections as well as for distance
computations.

The current geometric kernel provides two families of geometric objects:

one family is based on the representation of points using Cartesian coor-
dinates, the other family is based on the representation of points using
homogeneous coordinates. The homogeneous representation extends the
representation with Cartesian coordinates by an additional coordinate,
namely a common denominator. More formally, a point in d dimensional
space with homogeneous coordinates (x0 , x1 , . . . , xd−1 , xd ), xd 6= 0, has
Cartesian coordinates (x0 /xd , x1 /xd , . . . , xd−1 /xd ). This avoids divi-
sions and reduces many computations in geometric algorithms to cal-
culations over the integers. The homogeneous representation is used
for affine geometry in Cgal, and not projective geometry where the
homogeneous representation is usually known from. Both families are
parameterized by the number type used to represent the Cartesian or
homogeneous coordinates. The type CGAL::Cartesian<double>8 spec-
ifies the Cartesian representation with coordinates of type double, and
the type CGAL::Homogeneous<int> specifies the homogeneous repre-
sentation with coordinates of type int. These representation types are
used as template argument in all geometric kernel types, like a two-
dimensional point declared as

template <class R> CGAL::Point_2;

with a template parameter R for the representation class. Usually, type-

defs are used to introduce conveniently short names for the types. Here
is an example given for the point type with the homogeneous represen-
tation and coordinates of type int:
8 Cgal provides its own C++ namespace CGAL. All classes and functions of the

library are defined in this namespace and can be accessed via the scope prefix CGAL::.
72 Chapter 5. Computational Geometry Algorithms Library

typedef CGAL::Point_2< Homogeneous<int> > Point_2;

The class templates parameterized with CGAL::Cartesian or CGAL::

Homogeneous provide the user with a common interface to the under-
lying representation, which can be used in higher-level implementa-
tions independently of the actual coordinate representation. The list
of requirements on the template parameter defines the concept of a
representation class for the geometric kernel. Details for realizing this
parameterization can be found in [30, 31, 53].

Cgal provides clean mathematical concepts to the user without sac-

rificing efficiency. For example, Cgal strictly distinguishes points and
(mathematical) vectors, i.e. it distinguishes affine geometry from the
underlying linear algebra. Points and vectors are not the same as it is
discussed in [50] with regard to illicit computations resulting from iden-
tification of points and vectors in geometric computations. In particular,
points and vectors behave differently under affine transformations [104].
We do not even provide automatic conversion between points and vec-
tors but use the geometric concept of an origin instead. The symbolic
constant CGAL::ORIGIN represents a point and can be used to com-
pute the locus vector as the difference between a point and the origin.
Function overloading is used to implement this operation internally as
a simple conversion without any overhead. Note that we do not provide
the geometrically invalid addition of two points, since this might lead to
ambiguous expressions: assuming three points p, q, and r and an affine
transformation A, one can write in Cgal the perfectly legal expression
A(p + (q − r)). The slightly different expression A((p + q) − r) contains
the illegal addition of two points. Thinking in terms of coordinates, one
might expect the same result with the addition allowed as in the pre-
vious, legal expression. But this is not necessarily intended, since the
expression within the affine transformation would probably evaluate to
a vector, not a point as in the previous expression. Vectors and points
behave differently under affine transformations. To avoid these am-
biguities, the automatic conversion between points and vectors is not
provided.

Class hierarchies are used rarely in Cgal. An example are affine trans-
formations which maintain distinct internal representations specialized
on restricted transformations. The internal representations differ con-
siderably in their space requirements and the efficiency of their member
5.4. Geometric Kernel 73

functions. For all but the most general representation we gain perfor-
mance in terms of space and time. And for the most general repre-
sentation, the performance penalty caused by the virtual functions is
negligible, because the member functions are computationally expen-
sive for this representation. Alternatively we could have used this most
general representation for affine transformations only. But the use of
a hierarchy is justified, since the specialized representations, namely
translation, rotation and scaling, arise frequently in geometric comput-
ing.

Another design decision was to make the (constant-size) geometric

objects in the kernel non-modifiable. For example, there are no mem-
ber functions to set the Cartesian coordinates of a point. Points are
viewed as atomic units (see also [25]) and no assumption is made on
how these objects are represented. In particular, there is no assumption
that points are represented with Cartesian coordinates. They might use
polar coordinates or homogeneous coordinates instead. Then, member
functions to set the Cartesian coordinates are expensive. Nevertheless,
in current Cgal the types based on the Cartesian representation as
well as the types based on the homogeneous representation have both
member functions returning Cartesian coordinates and member func-
tions returning homogeneous coordinates. These access functions are
provided to make implementing own predicates and operations more
convenient.

Like other libraries [68, 13, 57] we use reference counting for the kernel
objects. Objects point to a shared representation and each represen-
tation counts the number of objects pointing to it. Copying objects
increments the counter of the shared representation, deleting an object
decrements the counter of its representation. If the counter reaches zero
by the decrement, the representation itself is deleted (see [70, Item 29]
for further information). The implementation of reference counting
is simplified by the non-modifiability of the kernel objects. However,
the use of reference counting was not the reason for choosing non-
modifiability. Using ‘copy-on-write’, i.e. a new representation is created
for an object whenever its value is changed by a modifying operation,
reference counting of modifiable objects is possible and only slightly
more involved. A comparison with a prototype of a geometric kernel
without reference counting can be found in [85]. The test applications
are two-dimensional convex hull algorithms. Reference counting costs
74 Chapter 5. Computational Geometry Algorithms Library

about 15% to 30% runtime for the types double and float, but it gains
2% to 11% runtime for the type leda real. Meanwhile, Cgal provides
two additional representation classes without reference counting. They
are named CGAL::Simple cartesian and CGAL::Simple homogeneous.

Further details of the geometric kernel can be found in [31], for exam-
ple the polymorphic behaviour of the return type of the intersection
functions. Recent developments towards an even more adaptable and
extensible geometric kernel are described in [53].

5.5 Basic Library

The basic library contains more complex geometric objects and data
structures, such as polygons, polyhedrons, triangulations (including
Delaunay, constrained, and regular triangulations), planar maps, range
and segment trees, and kd-trees. It also contains geometric algorithms,
such as convex hull, smallest enclosing circle, ellipse, sphere, and annu-
lus, polytope distance, boolean operations on polygons, and map over-
lay.

5.5.1 Generic Data Structures

Triangulations are an example of a container-like data structure in the

basic library. The interface contains, among others, member functions
to access the vertices of the triangulation. For example all vertices or
the vertices on the convex hull of the triangulation.

class Triangulation {
public:
Vertex_iterator vertices_begin();
Vertex_iterator vertices_end();

Convex_hull_iterator convex_hull_begin();
Convex_hull_iterator convex_hull_end();

// ...
};
5.5. Basic Library 75

The whole functionality for accessing vertices is factored out in separate

classes, which are models for the iterator concept.9

As in the previous example, geometric data structures in the basic

library often contain more than one sequence of interest, e.g. trian-
gulations contain vertices, edges, and faces. The names of the member
functions that return iterator ranges are prefixed with the name of the
sequence, e.g. vertices begin(), edges end(). These names are the
canonical extension of the corresponding names begin() and end() in
Stl. The iterator based interfaces together with the extended nam-
ing scheme assimilate the design of the container-like geometric data
structures in the basic library with the C++ Standard.

5.5.2 Generic Algorithms

An example of a geometric algorithm is the convex hull computation.

The algorithm takes a set of points and outputs the sequence of points
on the convex hull. The function declaration looks like this:

template < class InputIterator, class OutputIterator >

OutputIterator convex_hull( InputIterator first,
InputIterator beyond,
OutputIterator result);

Here, the input is read from the iterator range [first,beyond) and the
output is written to the output iterator result. Let the return-value
be result beyond, then the iterator range [result,result beyond)
contains the sequence of points on the convex hull. This design decou-
ples the algorithm from the container and gives the user the flexibility
to use any container, e.g. from Stl, from other libraries or own imple-
mentations (provided they are Stl compliant). It is even possible to
use no container at all, for example a sequence of points read from the
standard input:

9 The actual implementation in Cgal differs slightly, i.e. the vertices of the convex

hull of the triangulation are accessed with a more efficient circulator [58], since the
internal representation of this convex hull is cyclic.
76 Chapter 5. Computational Geometry Algorithms Library

convex_hull( istream_iterator<Point>( cin),

istream_iterator<Point>(),
ostream_iterator<Point>( cout, "\n"));

Points are taken from the standard input and the resulting points on the
convex hull are written to the standard output. Here so-called stream
iterators [72] from Stl are used. This example again demonstrates the
flexibility gained from the Stl-compliance of the geometric algorithms
in the basic library.

5.5.3 Traits Classes

An ideal theoretical paper on a geometric algorithm first declares geo-

metric primitives and thereafter expresses the algorithm in terms of
these primitives. Implementing an algorithm or data structure, we col-
lect all necessary types and primitive operations in a single class, called
traits class, which encapsulates details, such as the geometric represen-
tation. Collecting types in a single class is a template technique that is
already intensively used in [10]. It is sometimes called ‘nested typedefs
for name commonality’-idiom. The approach gains much additional
value by the traits technique as used in the C++ Standard Library [75],
where additional information is associated with already existing types
or built-in types.

An example is an iterator which refers to a particular value type. Algo-

rithms parameterized with iterators might need the value type directly.
This can be easily encoded as a local type for all iterators that are
implemented as classes:

struct iterator_to_int {
typedef int value_type;
// ...
};

Since a C-pointer is a valid iterator, this approach is not sufficient.

The solution chosen for Stl are iterator traits [75], i.e. class templates
parameterized with an iterator:
5.5. Basic Library 77

template < class Iterator >

struct iterator_traits {
typedef typename Iterator::value_type value_type;
// ...
};

The value type of the iterator example class above can be expressed as
iterator traits< iterator to int >::value type. For C-pointers
a specialized version of iterator traits exists, i.e. a class template para-
meterized with a C-pointer.

template < class T >

struct iterator_traits<T*> {
typedef T value_type;
// ...
};

Now the value type of a C-pointer, e.g. to int, can be expressed as

iterator traits< int* >::value type. This technique of providing
an additional, more specific definition for a class template is known as
partial specialization.

Our approach using traits classes in the basic library does not attach
information to built-in types, but to our data structures and algorithms.
We use them as a modularization technique that allows a single imple-
mentation to be interfaced to different geometric representations and
primitive operations. Our traits class is therefore a single template
argument for algorithms and data structures in the basic library, for
example triangulations:

template < class Traits >

class CGAL::Triangulation_2 {
// ...
};

Note that each primitive could as well be provided as a template para-

meter for itself, but using traits classes simplifies the interface. All
primitives are captured in a single argument, which makes it easier to
apply already prepared implementations of traits classes.

Traits classes must provide the geometric primitives required by the

geometric data structure or algorithm. Default implementations are
78 Chapter 5. Computational Geometry Algorithms Library

provided for the geometric kernel of Cgal. They are class templates
parameterized with a kernel representation class, for example CGAL::
Triangulation euclidean traits 2< CGAL::Cartesian<double> >.
A single traits class for triangulations is sufficient for all representations
and number types possible with the kernel. Further traits classes are
available in Cgal, for example for using the basic library with the
geometric part of Leda.

For algorithms implemented as functions, a default traits class is chosen

automatically if there is none given explicitly in the function call. Thus
the user can just ignore the traits class mechanism as in the following
example:

typedef CGAL::Cartesian<double> R;
typedef CGAL::Point_2<R> Point;
typedef CGAL::Polygon_2<R> Polygon;

Polygon hull;
CGAL::convex_hull_points_2( istream_iterator<Point>( cin),
istream_iterator<Point>( ),
back_inserter( hull));

In the call to the convex hull algorithm no traits class is visible to the
user. A default traits class is chosen automatically in the definition of
the algorithm:

template < class InputIterator, class OutputIterator >

OutputIterator
CGAL::convex_hull_points_2( InputIterator first,
InputIterator beyond,
OutputIterator result) {
typedef typename iterator_traits<InputIterator>::value_type
P;
typedef typename P::R R;
typedef CGAL::Convex_hull_traits_2<R> T;
return CGAL::convex_hull_point_2( first,beyond,result,T());
}

The value type of the iterator InputIterator is the point type used
in the input. It is determined with the iterator traits described previ-
ously. Since the default traits class is supposed to use the geometric
kernel of Cgal, we know that the point type must be a Cgal point
5.6. Conclusion 79

type and it ‘knows’ its representation type by means of a local type

named R. Finally, another version of CGAL::convex hull points 2 is
called with four arguments. The additional fourth argument is set to
CGAL::Convex hull traits 2<R>(), the default traits class for the con-
vex hull algorithms. This second version of the function template is
defined as follows:

template < class InputIterator,

class OutputIterator, class Traits >
OutputIterator
CGAL::convex_hull_points_2( InputIterator first,
InputIterator beyond,
OutputIterator result,
const Traits& traits) {
// compute the convex hull
// using only primitives from the traits class
}

5.6 Conclusion

We followed mainly the generic programming paradigm to achieve flex-

ibility and efficiency in Cgal. The compliance with Stl is important
in order to re-use its generic algorithms and container classes, and to
unify the look-and-feel of the design with the C++ standard. Cgal is
therefore easy to learn and easy to use for those who are familiar with
Stl.

The abstract concepts used in Stl are so powerful that only a few ad-
ditions and refinements are needed in Cgal. One refinement is the
concept of handles. Combinatorial data structures might not necessar-
ily possess a natural order on their items. Here, we restrict the concept
of iterators to the concept of handles, which is the item denoting part of
the iterator concept, and which ignores the traversal capabilities. Any
model of an iterator is a model for a handle. A handle is also known
as trivial iterator. Another refinement is the concept of circulators [58],
a kind of iterators with slightly adapted requirements to better suit the
needs of circular sequences. These occur naturally in several combina-
torial data structures, such as the sequence of edges around a vertex in
a triangulation.
80 Chapter 5. Computational Geometry Algorithms Library

The geometric traits classes offer great flexibility and modularity. There
is always a predefined traits class that uses types and operations of the
kernel. Where possible, this traits class is chosen by default, so the user
can totally ignore the existence of this mechanism.

In a few places we also used the object-oriented programming paradigm.

Examples are affine transformations and the polymorphic return type
of the intersection functions.
Chapter 6

A Quadratic
Programming Engine

This chapter presents the implementation of our algorithm for solving

linear and quadratic programs. We describe our design goals and the
design decisions made to achive these goals. The realization of the
solver and the basis inverse is discussed as well as the implementation
of different pricing strategies. We concentrate on the distinguishing
features of the code, while further details of the implementation and
the whole code can be found in [89, 90, 91].

6.1 Design Goals

The quadratic programming engine (QPE) is carefully designed and

implemented in C++. We follow the generic programming paradigm, as
it is realized in the Stl. Our design goals are closely related to those of
Cgal [31]. Among them, the most important ones are flexibility for the
user and efficiency of the code. For the QPE this means in particular:

1. Our method described in Chapter 2 solves quadratic programs

and also linear programs as a special case. If the user knows in
82 Chapter 6. A Quadratic Programming Engine

advance (i.e. at compile time), that the problem to solve has a

linear objective function, then (almost) no overhead should occur
compared to a stand-alone implementation of a solver for linear
programs.
2. Different geometric optimization problems come with different
representations. We want to allow the user to choose his favor-
ite format without the necessity of converting and copying his
problem to a specific representation. It should be also possible
to represent the possibly very large and dense objective matrix D
implicitly.
3. The correctness and efficiency of the algorithm heavily depends
on the underlying arithmetic. The user should be able to choose
a suitable (i.e. correctness guaranteeing) number type for the
internal computations on the one hand, while specifying the opti-
mization problem with one or several possibly different number
types on the other hand.
4. Since different optimization problems perform differently with dif-
ferent pricing strategies in general, there is no superior strategy
for all cases. The implementation should provide an easy-to-use
interface for choosing one of several predefined pricing strategies.
Furthermore, a framework for implementing own pricing strategies
should be available to the user.

6.2 Solver

The implementation of the QPE is divided in three parts: the solver [89],
the basis inverse [90], and the pricing strategies [91].

The solver is realized as a class template with a representation class as

template parameter.

template < class QPErep > class QPE_solver;

The template parameter QPErep is a traits class in the sense of Sec-

tion 5.5.3. In the sequel, we describe the requirements for QPErep.
6.2. Solver 83

6.2.1 Access to Original Problem

The QPE solves optimization problems with n variables and m con-

straints of the following form:
(QP) minimize cT x + xTD x
subject to A x S b (6.1)
x ≥ 0.
Here, A is an m×n-matrix, b an m-vector, c an n-vector, and D a pos-
itive semi-definite n×n-matrix. The symbol ‘S’ indicates that any of
the m order relations it stands for can independently be ‘≤’, ‘=’, or ‘≥’.
Compared to the general form given in (1.2) on page 16, the explicit
bounds on the variables are missing in (6.1). The current implementa-
tion of the QPE assumes nonnegativity constraints on the variables.

Different geometric optimization problems come with different repre-

sentations. We allow the user to choose his favorite format by only
asking for iterators to access the problem. This also avoids copying
overhead and, more important, gives the possibility of representing the
n×n-matrix D implicitly. This feature is already used in the solutions
for the polytope distance problem and the smallest enclosing ball prob-
lem, see next chapter.

We need five iterators for accessing A, b, c, D, and r. The latter repre-

sents the m order relations in (6.1). The iterator types are given in the
representation class:

struct QPErep {
typedef ... A_iterator;
typedef ... B_iterator;
typedef ... C_iterator;
typedef ... D_iterator;

enum Row_type = { EQUAL, LESS_EQUAL, GREATER_EQUAL };

typedef ... R_iterator;

// ...
};

The iterators for the actual problem are passed as parameters to the
solver’s set method, which is declared as follows:
84 Chapter 6. A Quadratic Programming Engine

template < class QPErep >

class QPE_solver {
public:
void set( int n, int m,
QPErep::A_iterator a_it, QPErep::B_iterator b_it,
QPErep::C_iterator c_it, QPErep::D_iterator d_it,
QPErep::R_iterator r_it);
// ...
};

All five iterators have to allow random-access, e.g. b it[j] is the j-th
entry of b and c it[i] is the i-th entry of c. Since A is accessed column-
wise, a it[j] is an iterator referring to the first entry in the j-th column
of A, while d it[i] is an iterator referring to the first entry in the i-th
row of D.

6.2.2 Tags for Special Properties

Some optimization problems have special properties, like a linear objec-

tive function or a symmetric objective matrix. If such a property is
known in advance, i.e. at compile time, the implementation of the QPE
can be tailored for this special case, thus making it more efficient as in
the general case. We support tags for the following three properties:

• The objective function is linear, i.e. D = 0.

• The objective matrix D is symmetric.
• The problem has no inequality constraints.

There are two types for defining tags, namely Tag true and Tag false.
If a property is present, the corresponding tag is defined to be of type
Tag true, otherwise of type Tag false.

struct QPErep {
// ...
typedef ... Is_linear;
typedef ... Is_symmetric;
typedef ... Has_no_inequalities;
// ...
};
6.2. Solver 85

In case the objective function is linear, the extension of the simplex

method to QP is switched off, resulting in almost no overhead com-
pared to a stand-alone implementation for LP. If the problem has only
equality constraints, the whole handling of slack variables is disabled.
A symmetric objective matrix halfs the number of accesses to entries
of D. All this is done at compile time, when the compiler only gener-
ates the code needed for the specific type of problem. We illustrate this
technique by the following example.

The current solution of the QPE consists of the values of the original
variables and the values of the slack variables. The latter are only
present, if the given problem has inequality constraints.

template < class QPErep >

class QPE_solver {
void compute_solution( )
{
// ...
// compute values of slack variables, if any
compute_slack_variables(
typename QPErep::Has_no_inequalities());
// ...
}
};

The function compute slack variables is called with an instance of

the type QPErep::Has no inequalities. At this point, the compiler
decides based on the actual type of the flag, which one of the two fol-
lowing implementations of compute slack variables has to be called.

template < class QPErep >

class QPE_solver {
void compute_slack_variables( Tag_true)
{
// nop
}

void compute_slack_variables( Tag_false)

{
// compute values of slack variables...
}
};
86 Chapter 6. A Quadratic Programming Engine

Thus, the code for computing the values of the slack variables is only
generated, if the given problem has inequality constraints.

6.2.3 Exact Arithmetic

The correctness and the efficiency of the algorithm heavily depend on

the underlying arithmetic. We rely on an exact number type for the
internal computations, which the user can choose independently from
the representation of the optimization problem. There only have to be
implicit conversions from the entry types of A, b, c, and D to the exact
number type.

struct QPErep {
typedef ... ET;
// ...
};

The arithmetic requirements for ET are quite basic. It has to support

addition, subtraction, and multiplication. A division operation is only
required for those cases where the remainder is zero. Fulfilling these
requirements, ET is a model for the concept RingNumberType of Cgal.

In Section 6.4.2 below, we describe how to combine the exact arithmetic

over ET with a very efficient floating point filter to speed up the pricing
step considerably.

6.2.4 Symbolic Perturbation

As introduced in Section 2.5 of Part I, we use symbolic perturbation

to cope with degenerate quadratic programs. The right-hand-side vec-
tor b is perturbed by the vector ε := (, 2 , . . . , m )T for some 0 <  < 1,
resulting in vector-valued polynomials in  as solutions of the quadratic
program.

During the ratio test, we test which basic variable becomes zero first,
when we increase the entering variable. This is done by finding the
smallest positive quotient ti = xi /qxi for all i ∈ B (cf. Section 2.3.2).
6.3. Basis Inverse 87

Now, the solution x∗B is a polynomial in , and so are the quotients:

xi ()
ti () =
qxi
x0 x1 x2 xm
= i + i  + i 2 + · · · + i m , i∈B. (6.2)
qxi qxi qxi qxi

The second equation is derived using (2.29) on page 35. Since 0 <  < 1,
we find the smaller of two ti () by comparing their first quotients as
defined in (6.2). Only if these have the same value, we compare their
second quotients, and so on. In other words, we compare two vectors
vi := (x0i /qxi , x1i /qxi , . . . , xm
i /qxi ) lexicographically. Because the first
entry of vi determines the sign of ti (), most comparisons are decided
after comparing the first entries. Usually, there are very few cases that
need the second or subsequent entries to decide the comparison.

Summarizing, the symbolic perturbation scheme does not introduce any

computational overhead in the current ratio test, if the non-perturbed
quadratic program has a unique minimal ti > 0. Only if the first entries
of the corresponding vectors vi have the same value, some additional
computations are needed. To this end, note that x1i , . . . , xm
i are just
entries of the corresponding row of the basis inverse. This can be seen
by replacing b with b+ε in (2.7) on page 25.

The same technique is used to determine tj with µj (tj ) = 0 and to com-

pare it with the smallest positive ti .

6.3 Basis Inverse

Given a basis B, the basis matrix is defined as

!
0 AE ∪S
˙ N,BO
MB := , (6.3)
ATE ∪S
˙ N,BO 2DBO,BO

and MB−1 is called the basis inverse. From our generalized simplex
method used in the QPE, we have the following additional structure
and requirements.
88 Chapter 6. A Quadratic Programming Engine

• We do not handle just one basis matrix MB , but a sequence of

matrices. Here, successive ones only differ slightly, i.e. one row
and/or one column is either appended, removed, or replaced. We
want to exploit this coherence.

• If the matrix MB and the vectors bE ∪S˙ N and −cBO contain inte-
gral entries, then the entries of the solution vectors x∗BO and λ
are rational, and we would like to obtain exact rational represen-
tations of them. This means that explicit divisions have to be
avoided during the solution process (unless they do not leave a
remainder).

• If D is the zero-matrix, i.e. the problem to solve is a linear pro-

gram (LP), we want to have only a small overhead in time and
space compared to a stand-alone implementation of the simplex
method for solving LPs.

We will refer to the QP case or the LP case in the sequel, if the problem
to solve is a quadratic or linear program, respectively.

6.3.1 Rational Representation

The objective function does not change, if we vary a pair of elements

of D symmetric to the main diagonal, as long as its sum remains the
same. Thus, we may assume w.l.o.g. that D is symmetric, and so are MB
and MB−1 . Consequently, we will only store the entries on and below
the main diagonal.

In the LP case, every basis has size m. The basis matrix assumes the
form !
0 AE ∪S
˙ N,BO
MB = ,
ATE ∪S
˙ N,BO 0

with |E| + |SN | = |E| + |S| − |SB | = m − |BS | = |B| − |BS | = |BO |,
˙ N,BO is quadratic. The resulting basis inverse is
i.e. AE ∪S

−1 T!
0 (AE ∪S
˙ N,BO )
MB−1 = −1
(6.4)
˙ N,BO )
(AE ∪S 0
6.3. Basis Inverse 89

−1
˙ N,BO )
and it suffices to store (AE ∪S , avoiding any space overhead com-
pared to a stand-alone implementation of the simplex method for solving
linear programs.

To address the second requirement, we consider Cramer’s well-known

rule for the inverse of a matrix M in terms of the M ji ,

−1 (−1)i+j det(M ji )
Mi,j = .
det(M )

It follows that the entries of M −1 can be written as rationals with a com-

mon denominator det(M ), if M contains integral entries. Hence, M −1
can also be stored with integral entries, keeping the denominator sepa-
rately. For practical reasons, we prefer the absolute value |det(M )| to
the signed value det(M ) as the common denominator. Then, for exam-
ple, the numerators appearing in the matrix-vector products already
have the same sign as the true rational values. We store M −1 in the
form
−1 sgn(det(M )) (−1)i+j det(M ji )
Mi,j = . (6.5)
| det(M )|
The integral part of the basis inverse is defined by

M̂ −1 := d M −1 ,

where d := |det(M )|. In the sequel, any value x and the corresponding x̂
satisfy x = x̂/d.

6.3.2 Updates

The QPE produces a sequence of bases, where successive ones only

differ by one variable. Either a nonbasic variable enters the basis
(QP case), a basic variable leaves the basis (QP case), or a nonbasic
variable replaces a basic variable in the basis (LP case). Since we dis-
tinguish between original and slack variables, we have the following eight
different types of updating the basis inverse:

U1 An original variable enters the basis, i.e. BO is increased by one

element.
90 Chapter 6. A Quadratic Programming Engine

U2 An original variable leaves the basis, i.e. BO is decreased by one

element.
U3 A slack variable enters the basis, i.e. SN is decreased by one
element.
U4 A slack variable leaves the basis, i.e. SN is increased by one ele-
ment.
U5 An original variable replaces an original variable in the basis,
i.e. one element in BO is replaced.
U6 A slack variable replaces a slack variable in the basis, i.e. one
element in SN is replaced.
U7 An original variable replaces a slack variable in the basis, i.e. BO
and SN each increase by one element.
U8 A slack variable replaces an original variable in the basis, i.e. BO
and SN each decrease by one element.

Note, the first four update types belong to the QP case, while the latter
four belong to the LP case.

An update of type U1, U4, or U7 enlarges the matrix M by one row and
one column. We may assume w.l.o.g. that the row and the column are
appended to the bottom and to the right of M , respectively, to simplify
the presentation. Let (uT, w) be the row and (v T, w)T the column to
add, with w being the entry they have in common. Then the new matrix
is  
M v
M> =  .
 
T
u w
It is easy to verify that
   
1 0 0 1
 ... .. .. ..
.

M .
 . y 
M> =  (6.6)
   
1 0 0 1
  
   
xT 1 0 ... 0 z 0 ... 0 1
holds, where
xT := uT M −1 , y := M −1 v, z := w − uT M −1 v.
6.3. Basis Inverse 91

This implies for the inverse of the new matrix

   
1 0 1 0
 . . . −y   −1
..   . . ..
−1  M .  . .

M> =
 
1 0  1 0
  
  
T
0 ... 0 1 0 . . . 0 1/z −x 1
 
1 z M −1 + y xT −y 
= . (6.7)
z

−xT 1

Since we represent M −1 in rational form, also xT , y, and z are repre-

sented as (vectors of) rationals with common denominator d, i.e.

x̂T ŷ ẑ
xT =: , y =: , z =: . (6.8)
d d d
From (6.6) we get

det(M> ) = det(M ) z = sgn(det(M )) d z = sgn(det(M )) ẑ,

hence |ẑ| is the denominator of the rational representation of the inverse

of the new matrix. Substituting (6.8) in (6.7) gives
 
−1 d (ẑ M̂ −1 + ŷ x̂T )/d2 −ŷ/d 
M> =
ẑ
 
−x̂T/d 1
 
sgn(ẑ) (ẑ M̂ −1 + ŷ x̂T )/d −ŷ 
= (6.9)
|ẑ|
 
−x̂T d
−1
M̂>
= .
|ẑ|

Note, the division by d is without remainder, following from Cramer’s

rule.

The complementary operation, i.e. the removal of one row and one
column, is done by updates of type U2, U3, and U8. We assume
w.l.o.g. that the last row and the last column of M should be removed,
92 Chapter 6. A Quadratic Programming Engine

to simplify the presentation. This can always be achived by swapping

the row and the column to be removed with the last row and the last
column, respectively. Now we look at the last row and the last column
as if they were just appended to M as described above. Equation (6.9)
holds by the uniqueness of the inverse, and we get
 
 . . . −ŷ  −1
sgn(ẑ)   = M̂ (6.10)
T
−x̂ d

with common denominator |ẑ|. The integral part of the new inverse
−1
M< is obtained by
   
−1 .. .. 
M̂< .  1  ŷ x̂T . 

= sgn(ẑ) d M̂ −1 −   , (6.11)
ẑ

... . ... .

ignoring the last row and the last column of M̂ −1 in the calculation
above. Again, the division by ẑ is without remainder, and the new
denominator is d = |det(M< )|. The sign of ẑ can be computed easily
using the fact that d has to be positive by definition.

The two remaining update types either replace one row (U5) or one
column (U6) of M . We can assume w.l.o.g. that the last row or column
is replaced. Let uT be the new row and v the new column, respectively.

We define xT := uT M −1 and y := M −1 v and get new matrices Mr (row

replaced) and Mc (column replaced) as
 
1 0
 ... .
.. 
Mr =  M (6.12)
 
 1 0 
x1 . . . xk−1 xk

and
 
1 y1
.. .. 
 . . 
Mc = M  . (6.13)

 1 yk−1 
0 . . . 0 yk
6.3. Basis Inverse 93

The inverses of the new matrices are

 
1 0
.. ..
−1 −1 1  . .

Mr = M
 
xk  1 0
 

−x1 . . . −xk−1 1

and
 
1 −y1
.. .. 
1  . . 
Mc−1 =  M −1 .

yk 1 −yk−1 


0 ... 0 1

By substituting M −1 = M̂ −1/d, xT = x̂T/d, and y = ŷ/d, we obtain the

integral parts as
 
1 0
.. ..
M̂ −1 d  . .

Mr−1 =
 
d x̂k  1 0
 

−x̂1 /d . . . −x̂k−1 /d 1
 
d 0
.. ..
sgn(x̂k )   . .
 M̂ −1
= M̂ −1   /d = r (6.14)
 
|x̂k |  d 0  |x̂k |
−x̂1 . . . −x̂k−1 d

and
 
1 −ŷ1 /d
.. ..  M̂ −1
d  . .
Mc−1 =
 
ŷk  1 −ŷk−1 /d d
 
0 ... 0 1
 
d −ŷ1
.. .. 
sgn(ŷk )  . . 
 M̂ −1
=  M̂ −1 /d = c . (6.15)

|ŷk | d −ŷk−1  |ŷk |


0 ... 0 d
94 Chapter 6. A Quadratic Programming Engine

The new denominators are |x̂k |, since det(Mr ) = xk det(M ) = x̂k holds
by (6.12), and |ŷk |, since det(Mc ) = yk det(M ) = ŷk holds by (6.13).

Updates of the first four types can be performed in time O((m + |BO |)2 )
and updates of the latter four types in time O(m2 ). This is substantially
cheaper than computing the basis inverse from scratch.

The technique of updating the basis inverse when a column in the basis
matrix is replaced has been proposed before by Edmonds and termed
‘Q-pivoting’ [28]. It is also used, for example, by Gärtner in his exact
implementation of the simplex method [39] and by Avis in his vertex
enumeration algorithm [5].

6.3.3 Implementation

The basis inverse is realized in the class template QPE basis inverse.
It is parameterized with an exact number type ET and a compile time
tag IsLP. The latter indicates whether the problem to solve is a linear
program.

template < class ET, class IsLP > class QPE_basis_inverse;

The solver of the QPE contains a basis inverse as data member.

template < class QPErep >

class QPE_solver {
// ...
QPE_basis_inverse<typename QPErep::ET,
typename QPErep::Is_linear> inv_M_B;
// ...
};

As in the implementation of the solver, the tag IsLP lets the compiler
decide which code to generate for the basis inverse, resulting in an effi-
cient tailored implementation for the specific problem to solve.

Following the rational representation (6.5), we store the integral part of

the basis inverse as a vector of rows, each row being a vector of entries,
representing the numerators. The denominator is kept separately.
6.4. Pricing Strategies 95

template < class ET, class IsLP >

class QPE_basis_inverse {
// ...
std::vector< std::vector<ET> > M; // numerators
ET d; // denominator
// ...
};

From the description of the update types we know, that in the QP case
the index sets BO and E ∪˙ SN can increase and decrease independently.
This means that the number of rows (columns) in the submatrix of A
(AT ) can increase and decrease. To handle this, one could append new
rows and columns always at the lower right part of the matrix, and
fill up a removed row and column by the last row and column of the
matrix, respectively. The major drawback of this approach is, besides
the additional work for the swapping, that the index sets get mixed up
after some updates, which makes multiplying vectors with the matrix
complicated and expensive. We want to keep the rows and columns
corresponding to the same index set together, without having to move
or copy parts of the basis inverse when rows are added to the submatrix
of A. Therefore we must make sure that enough space is available in the
upper left part of the matrix. Fortunately, the size of E ∪˙ SN is bounded
by l := min{n, m} (cf. Section 2.4), so we are able to reserve enough
space for the upper left part at once when the matrix is initialized, see
Figure 6.1.

A full description of the basis inverse implementation can be found

in [90].

6.4 Pricing Strategies

Pricing is the process of finding the entering variable, i.e. a nonbasic

variable xj with µj < 0 (cf. Section 2.3.1), where µj is defined as

µj = cj + ATj λ + 2DB,j
T
x∗B .

Taking inequality constraints into account (cf. Section 2.4), we get

∗
µj = cj + ATE ∪S
˙ N ,j λE ∪S
T
˙ N + 2DBO ,j xBO .
96 Chapter 6. A Quadratic Programming Engine

column 0 column l
↓ ↓
···
row 0 → m1,1 . . . m1,s m1,l+1 ... m1,l+b
.. .. .. .. →
. . . . ←

ms,1 . . . ms,s ms,l+1 ... ms,l+b

&
- ↓↑
···
row l → ml+1,1 . . . ml+1,s ml+1,l+1 . . . ml+1,l+b

.. .. → .. ..
. . ← . .

ml+b,1 . . . ml+b,s ml+b,l+1 . . . ml+b,l+b

↓↑ &
-
.. ..
. .

Figure 6.1: Memory layout of basis inverse, s := |E ∪˙ SN |, b := |BO |

Since the basis inverse has a rational representation, so has the current
∗
solution, and the entries of the vectors λE ∪S
˙ N and xBO are quotients
with common denominator d. We obtain
∗
µ̂j = d cj + ATE ∪S
˙ N ,j λ̂E ∪S
T
˙ N + 2DBO ,j x̂BO , (6.16)
∗ ∗
where λ̂E ∪S
˙ N and x̂BO contain the numerators of λE ∪S ˙ N and xBO ,
respectively. The values µj and µ̂j agree in sign because d is positive
by definition.

Usually many nonbasic variables qualify for entering the current basis,
in which case we have the freedom to choose one of them. The actual
choice is done by the pricing strategy according to some pivot rule.
6.4. Pricing Strategies 97

6.4.1 Partial Pricing

Testing all nonbasic variables in one iteration of the pivot step can be
very expensive, if the set of nonbasic variables N is large. This is the
case for optimization problems with high variables-to-constraints ratio,
which we have in our applications. The idea of partial pricing is to
maintain a set S of active variables, which is initially relatively small
(see below). The entering variable is chosen from the variables in S.
Only if no active variable qualifies for entering the current basis, the
remaining nonbasic variables are tested.

Define min(R) to be the first index j with xj ∈ R and µj < 0 for an

ordered set R ⊆ N .

Algorithm 6.1 (returns an entering variable xj or optimal)

PartialPricing(S):
j := min(S)
IF µj < 0 THEN
RETURN xj
ELSE
V := {k ∈ N \S | µk < 0}
IF V = ∅ THEN
RETURN optimal
ELSE
S := S ∪ V
RETURN min(V )
END
END

When the current basis is updated, the entering variable xj is removed

from S. Any variable leaving the basis is appended to S.

In the LP case, the choice of min(S) (respectively min(V )) as the enter-

ing variable is known as Dantzig’s rule. Here, the idea is that variables
with negative but large absolute µj yield a fast decrease in the objective
function’s value, because such variables violate the KKT conditions by
a larger ‘amount’ than others.

The intuition behind partial pricing is that S and V are always small
and that S is augmented only a few times. In this case, most pricing
98 Chapter 6. A Quadratic Programming Engine

steps are cheap, because they operate on a small set of active variables.
Furthermore, only a few runs through the whole set N of nonbasic vari-
ables to find the set V are needed. Exactly the same intuition lies behind
Clarkson’s LP algorithm [21]. It works in the dual setting (i.e. with
few variables and many constraints) and can easily be formulated as
a dual simplex method. The interpretation of Clarkson’s algorithm as
a dual partial pricing scheme has already been suggested by Adler and
Shamir [1].

A theoretical analysis of partial pricing in the LP case has been done by

Gärtner and Welzl [47]. They showed the following under the assump-
p
tion of a nondegenerate linear program: if we choose |S| = m n/2,
then S is augmentedpat most m times and the expected size of S is
bounded by 2(m+1) n/2. Although this assumption does not always
hold in practice, we nevertheless keep applying the
p partial pricing strat-
egy and find that it still works well. We use m n/2 as the initial size
of S in our implementation.

6.4.2 Filtered Pricing

Another way of speeding up the pricing step is to use fast floating point
arithmetic instead of exact arithmetic. The following scheme is based
on a similar scheme proposed by Gärtner [39] for linear programs, which
we adapted to our quadratic programming solver.

Instead of evaluating (6.16) exactly, we compute floating point approx-

imations
µ̃j = (d˜ ⊗ cj ) ⊕ (ATE ∪S T ∗
˙ N ) ⊕ (2DBO ,j  x̃BO ) ,
˙ N ,j  λ̃E ∪S (6.17)
where λ̃E ∪S ∗ ˜
˙ N , x̃BO , and d are nearest floating point approximations to
∗
λ̂E ∪S
˙ N , x̂BO , and d, respectively. They are computed once in the begin-
∗
ning of the pricing step. In case d or one of the entries of λ̂E ∪S
˙ N or x̂BO
is larger than the largest representable floating point number, λ̂E ∪S ˙ N,
x̂∗BO , and d are scaled by a suitable power of two in advance. The oper-
ators ⊕, ⊗, and  denote the floating point addition, multiplication,
and inner product, respectively.

Using Dantzig’s rule, the obvious candidate for the entering variable is
the nonbasic variable xj with smallest value µ̃j . For the correctness of
6.4. Pricing Strategies 99

the algorithm, it does not matter whether xj = min(N ) really holds,

i.e. if we found the ‘best’ entering variable according to the rule. The
important property of xj is that µ̂j is negative, which can be easily
checked using exact arithmetic. The benefit of Dantzig’s rule in our
context is that µ̂j is very likely to be negative, because it has been found
to be the smallest among all, with floating point arithmetic computed
values.

If the exact check of µ̂j succeeds, xj is the entering variable. Otherwise

we have to verify, that really no improving variable exists. The straight-
forward solution would be to recompute all µ̂j s with exact arithmetic to
check whether an improving variable has been missed due to rounding
errors in the floating point computations. This would be quite expen-
sive and is not necessary in most cases. We already know that all inex-
act values µ̃j , j ∈ N are nonnegative, and a candidate for the entering
variable has been missed if and only if some exact value µ̂j is nega-
tive. Lemma 6.2 below gives two error bounds on µ̃j that let us deduce
µ̃j ≥ 0 in case µ̃j is sufficiently far above zero. Only the variables that
cannot be decided using these error bounds need to be dealt with exact
arithmetic. The hope is that most values are ‘caught’ by this high-level
floating point filter, which is usually the case in our applications.

Lemma 6.2 (Gärtner [39]) Let

n
R0 := max |ci | ,
i=1
n
RkA := max |Ak,i | , k ∈ [m],
i=1
n
RkD := max |2Dk,i | , k ∈ [n],
i=1
m n
Cj := max{ |cj | , max |Ai,j | , max |2Di,j | } , j ∈ [n],
i=1 i=1

be the row and column maxima of the quadratic program. Define

U := max{ d˜ ⊗ R0 , max ( |λ̃i | ⊗ RiA ) , max ( |x̃∗i | ⊗ RiD ) } ,
˙ N
i∈E ∪S i∈BO

W := max{ d˜, max |λ̃i | , max |x̃∗i | } .

˙ N
i∈E ∪S i∈BO

If the floating point arithmetic has p bits of precision, then

|µ̃j − µ̂j | ≤ min{ U ⊗ q , W ⊗ q ⊗ Cj }, j∈N, (6.18)

where q = (1 + 1/64) (|E ∪˙ SN |+|BO |+1) (|E ∪˙ SN |+|BO |+2) 2−p .

100 Chapter 6. A Quadratic Programming Engine

Note that the bounds in (6.18) can be exactly evaluated with floating
point arithmetic. This is important, because otherwise rounding errors
could occur in computing the error bounds. Since q is quite small, the
bounds are usually very good. A good choice for the floating point arith-
metic is the type Double as defined in the IEEE standard 754 for binary
floating-point arithmetic [55], which has p = 53 bits of precision. In C++
it is realized as the type double, which we use in our implementation
of filtered pricing.

Proof. Let x and y be vectors of length l and let u := 2−p be the

unit roundoff of the floating point arithmetic, also known as machine
epsilon. Classical results of Forsythe and Moler [34] give

x̃i = xi (1+δi ) , xi = x̃i (1+εi ) , x̃i yi = x̃i ⊗ yi (1+ηi ) ,

with |δi | ≤ u, |εi | ≤ u, and |ηi | ≤ u for i ∈ [l]. Furthermore, if lu < 0.01,
then
Xl
x̃T  y = x̃i yi (1+1.01 lΘi u)
i=1

holds, with |Θi | ≤ 1 for i ∈ [l]. Using this, the error of the inner product
in floating point arithmetic can be estimated as
Xl 
| x̃T  y − xT y | =  (xi yi (1+δi )(1+1.01 lΘi u) − xi yi )
 
i=1
Xl 
= x̃i yi (1+εi )(1.01 lΘi u + δi + 1.01 lδi Θi u)
 
i=1
Xl 
≤ x̃i yi  (1+u)(1.01 lu + u + 1.01 lu2 )
 
i=1

l
≤ max |x̃i yi | (1+u)(1.01 l2 u + lu + 1.01 l2 u2 )
i=1

l
≤ max |x̃i ⊗ yi | (1+u)2 (1.01 l2 u + lu + 1.01 l2 u2 )
i=1

l
≤ max |x̃i ⊗ yi | 1.01 l (l+1) u .
i=1

The last inequation holds for any practical value of l, if u = 2−53 .

6.4. Pricing Strategies 101

We bound the maximum from above in two ways. Firstly, we have

l l
max |x̃i ⊗ yi | ≤ max( |x̃i | ⊗ max |yi | ) ,
i=1 i=1 y

where y runs over all vectors we consider during the pricing. Secondly,
we get

l l l
max |x̃i ⊗ yi | ≤ max |x̃i | ⊗ max |yi | .
i=1 i=1 i=1

Majorizing the constant 1.01 by (1+1/64) yields a bound exactly com-

∗T
putable with floating point arithmetic. Defining xT := (d, λ̂TE ∪S
˙ N , x̂BO )
T T T
˙ N ,j , 2DBO ,j ) completes the proof.
and y := (cj , AE ∪S 2

We apply the bounds of the lemma in the following way. At first, we

check whether µ̃j ≥ U ⊗ q holds. If not, we test µ̃j against the second
bound W ⊗ q ⊗ Cj , which requires one additional multiplication per
variable. Only in case the second test also fails, we have to resort to
exact arithmetic to determine the sign of µ̂j .

6.4.3 Implementations

The whole pricing step is encapsulated in a class. It can be replaced with

other pricing strategies derived from a given base class. This feature
has been used to perform the tests involving full and partial pricing and
different arithmetic, see Chapter 8.

The class template QPE pricing strategy is the base class for all pric-
ing strategies. It is parameterized with the representation class QPErep
of the ambient solver and optionally with a fast and possibly inexact
number type used in the filtered pricing.

template < class QPErep, class NT = double >

class QPE_pricing_strategy {
protected:
const QPE_solver<QPErep>* solverP;
// ...
};
102 Chapter 6. A Quadratic Programming Engine

The pointer to the ambient solver gives access to the original problem
and the current solution. The variable solverP is set, when the pricing
strategy becomes the current pricing strategy of the solver.

The member function pricing performs the actual pricing step. It is

called by the ambient solver and returns either the index of the entering
variable or −1 to indicate optimality.

template < class QPErep, class NT >

class QPE_pricing_strategy {
public:
virtual int pricing( ) const = 0;
// ...
};

Since pricing is declared pure virtual, it has to be reimplemented by

the derived pricing strategy classes.

Any pricing strategy has to check the sign of µj for all nonbasic variables,
or at least for a subset of them. We provide two member functions
which compute µ̂j and µ̃j , respectively. One uses exact arithmetic over
the number type QPErep::ET, the other one uses the fast and possibly
inexact number type NT (as described in the previous section).

template < class QPErep, class NT >

class QPE_pricing_strategy {
protected:
typedef typename QPErep::ET ET;
ET mu ( int j) const;
NT mu_NT( int j) const;
// ...
};

These member functions are called several times during the pricing step.
Thus, we provide a member function init NT that computes the floating
point approximations λ̃E ∪S ∗ ˜
˙ N , x̃BO , and d and stores the values for later
use by the member function mu NT. Pricing strategies using this function
have to call init NT at the beginning of their pricing member function.

Some pricing strategies maintain an internal status, e.g. the set of active
variables in case of partial pricing. The necessary initialization can be
6.4. Pricing Strategies 103

done in a reimplementation of the init member function, which is called

by the ambient solver. At the end of the pricing member function, the
pricing strategy knows the entering variable, but what about leaving
variables? The member function leave basis is called by the ambient
solver with the corresponding index each time a variable leaves the basis.
In addition, the pricing strategy is notified by a call to the transition
member function if the solver switches from phase I to phase II.

template < class QPErep, class NT >

class QPE_pricing_strategy {
public:
virtual void init( ) { }
virtual void transition( ) { }
virtual void leave_basis( int i) { }
// ...
};

The default implementations of init, transition, and leave basis do

nothing, but they can be reimplemented by the derived pricing strategy
classes.

We provide predefined pricing strategies for all four combinations of

full/partial pricing and exact/filtered pricing:

template < class QPErep > class QPE_full_exact_pricing;

template < class QPErep > class QPE_partial_exact_pricing;
template < class QPErep, class NT = double >
class QPE_full_filtered_pricing;
template < class QPErep, class NT = double >
class QPE_partial_filtered_pricing;

In the sequel, we describe, as an example, the implementation of the

most sophisticated and usually most efficient strategy, namely the com-
bination of partial and filtered pricing.

The pricing starts with finding the active variable xj ∈ S with smallest
value µ̃j using floating point arithmetic. If the exact check µ̂j < 0 suc-
ceeds, xj is returned. Otherwise the smallest µ̃j among the remaining
nonbasic variables is computed using floating point arithmetic a second
time. Again, if the exact check µ̂j < 0 succeeds, xj is the entering vari-
able. In this case, S is augmented with V , i.e. all nonbasic variables
104 Chapter 6. A Quadratic Programming Engine

with µ̃j < 0 found during the last scan are added to the set of active
variables. If no entering variable has been found so far, we have to verify
that really no entering variable exists. For each nonbasic variable, we
check µ̃j against the two bounds of Lemma 6.2. If this does not decide
the sign of µ̂j , we use exact arithmetic to determine it. In case a nega-
tive µ̂j is found, xj is returned, otherwise optimality has been certified.
The following pseudocode describes the partial pricing strategy.

Algorithm 6.3 (returns an entering variable xj or optimal)

PartialFilteredPricing(S):
j := arg min{ µ̃k | k ∈ S }
IF µ̂j < 0 THEN
RETURN xj
ELSE
V := {k ∈ N \S | µ̃k < 0}
IF V 6= ∅ THEN
j := arg min{ µ̃k | k ∈ V }
IF µ̂j < 0 THEN
S := S ∪ V
RETURN xj
END
ELSE
FOREACH k IN N DO
IF (NOT µ̃k ≥ U ⊗ q) AND (NOT µ̃k ≥ W ⊗ q ⊗ Ck ) THEN
IF µ̂k < 0 THEN
RETURN xj
END
END
END
RETURN optimal
END
END

The approximate values µ̃k and the exact values µ̂k are computed using
the base class’ member functions mu NT and mu, respectively.

To apply the error bounds of the floating point filter, we need the row
and column maxima of the quadratic program as defined in Lemma 6.2.
6.5. Conclusion 105

Computing these maxima completely during the initialization of the

pricing strategy would require to access all entries of A and D at least
once, resulting in O(mn+n2 ) initialization time. Instead, we use a
variant of lazy evaluation, i.e. only the needed maxima are computed
respectively updated ‘on-the-fly’ in each pricing step. Initially, only R0
is computed while the other row maxima RkA and RkD are set to zero.
The column maxima Cj are initialized with |cj | for j ∈ [n].

Just before the FOREACH loop in Algorithm 6.3, we compute RkA for
each k ∈ E ∪˙ SN and RkD for each k ∈ BO . The Cj s are updated with the
absolute values of A’s rows in E ∪˙ SN and D’s rows in BO . To avoid
repeated computations of the same values, we store all maxima com-
puted so far and keep track of the rows already handled. This scheme
avoids unnecessary computations, thus providing the needed maxima
efficiently at minimal cost.

Further details of the implementations can be found in [91].

6.5 Conclusion

We presented a C++ implementation of our quadratic programming algo-

rithm. It was carefully designed to be efficient and easy to use at the
same time. We reached our design goals by using generic programming
with class templates, compile time tags and iterators. The flexibility
for the pricing strategy was achieved through an object-oriented design
with a base class and virtual functions. The combination of exact and
floating point arithmetic in the partial filtered pricing strategy leads to
a considerable speed-up.

A preliminary version of the quadratic programming engine is already

a (hidden) part of Cgal. It is used to solve geometric quadratic pro-
gramming problems, see the next chapter.
Chapter 7

Geometric Optimization
Problems in Cgal

This chapter describes the implementations of some geometric optimi-

zation problems in Cgal, namely the first three problems presented in
Chapter 3.

7.1 Polytope Distance

7.1.1 Definition

An object of the class Polytope distance d<Traits> represents the

(squared) distance between two convex polytopes, given as the con-
vex hulls of two finite (multi)sets of points in d-dimensional Euclidean
space Ed . For point sets P and Q we denote by pd(P, Q) the distance
between the convex hulls of P and Q. Note that pd(P, Q) can be de-
generate, i.e. pd(P, Q) = ∞ if P or Q is empty.

We call two inclusion-minimal subsets SP of P and SQ of Q with

pd(SP , SQ ) = pd(P, Q) a pair of support sets, the points in SP and SQ
are the support points. A pair of support sets has size at most d+2
108 Chapter 7. Geometric Optimization Problems in Cgal

(by size we mean |SP | + |SQ |). The distance between the two polytopes
is realized by a pair of points p and q lying on the convex hull of SP
and SQ , respectively, i.e. ||p − q|| = pd(P, Q). In general, neither the
support sets nor the realizing points are necessarily unique.

The underlying solver can cope with all kinds of input, e.g. P and Q
may be in non-convex position or points may occur more than once.
The algorithm computes a pair of support sets SP and SQ and the
corresponding realizing points p and q.

7.1.2 Solution

Given the two point sets as P = { p1 , . . . , pr } and Q = { q1 , . . . , qs }

with r+s = n, let x∗ = (x∗1 , . . . , x∗n ) be an optimal solution to

(PD) minimize xT C T C x
Pr
subject to xi = 1
Pi=1
n (7.1)
i=r+1 xi = 1
x ≥ 0,

with C = (p1 , . . . , pr ,−q1 , . . . ,−qs ). Then the support sets are deter-
mined by the positive x∗i s, namely

SP = {pi ∈ P | x∗i > 0} ,

SQ = {qi ∈ Q | x∗r+i > 0} .

The realizing points are convex combinations of the given points, i.e.
r
X
p= x∗i pi ,
i=1
s
X
q= x∗i+r qi .
i=1

7.1.3 Implementation

Matrix A in (7.1) has two rows of length n. The first row contains
a block of r ones, followed by a block of s zeros. The second row contains
7.1. Polytope Distance 109

the same blocks with ones and zeros swapped. Since A is accessed
column-wise, we store it as a vector of C-arrays of length 2. Since b
is the 1-vector and c the 0-vector, we represent them with a special
iterator referring always to the same constant value.

The interesting part is the representation of the objective matrix. Each

entry of D is an inner product of two input points, in particular


 pTi pj if 1 ≤ i ≤ r , 1 ≤ j ≤ r
−pTi qj−r if 1 ≤ i ≤ r , r < j ≤ n

Di,j = (7.2)
−q T pj if r < i ≤ n , 1 ≤ j ≤ r
 T i−r


qi−r qj−r if r < i ≤ n , r < j ≤ n

by the definition of C in (7.1). The idea for the implicit representation

of D is as follows. If the entry in the i-th row and the j-th column is
accessed, the corresponding input points are multiplied ‘on-the-fly’ and
the signed result is returned.

We store matrix C in the variable points, which is a vector of signed

points, thus allowing random-access to the input points. The class tem-
plate PD D iterator is initialized with the iterator points.begin()
referring to the first input point in C. When dereferenced at posi-
tion i, it returns an object of type PD D row iterator initialized with
the index i and the iterator points.begin(). This object is another
iterator implicitly representing the i-th row of D. When dereferenced
at position j, it accesses the points at positions i and j, computes their
inner product, and returns the result.

template < class Point >

class PD_D_row_iterator {
// types
typedef typename std::vector<Point>::const_iterator
Point_it;
typedef ... CT; // points’ coordinate type

// data members
Point_it points; // iterator to C
int i; // row index
int j; // column index

public:
PD_D_row_iterator( Point_it it, int row)
110 Chapter 7. Geometric Optimization Problems in Cgal

: points( it), i( row), j( 0) { }

CT operator * ( )
{ return inner_product( points[ i], points[ j]); }
// ...
};

template < class Point >

class PD_D_iterator {
// types
typedef typename std::vector<Point>::const_iterator
Point_it;
typedef PD_D_row_iterator<Point> Row_it;

// data members
Point_it points; // iterator to C
int i; // row index

public:
PD_D_iterator( Point_it it) : points( it), i( 0) { }
Row_it operator * ( ) { return Row_it( points, i); }
// ...
};

The primal version of the polytope distance problem we use here has
the special properties of a symmetric objective matrix and no inequality
constraints. The following class template collects all types needed by
the QPE to solve (7.1).

template < class Traits, class ET_ >

class PD_rep {
typedef ... Point; // point type (from Traits)
typedef ... CT; // points’ coordinate type
public:
typedef std::vector< CT[ 2] >::const_iterator A_iterator;
typedef Const_value_iterator< CT > B_iterator;
typedef Const_value_iterator< CT > C_iterator;
typedef PD_D_iterator< Point > D_iterator;

enum Row_type { EQUAL };

typedef Const_value_iterator< Row_type > R_iterator;

typedef Tag_false Is_linear;

typedef Tag_true Is_symmetric;
typedef Tag_true Has_no_inequalites;
7.2. Smallest Enclosing Ball 111

typedef ET_ ET;

};

An implementation of the dual version of the polytope distance problem

has been done for the experiments described in the next chapter. It will
become part of Cgal in a future release.

7.2 Smallest Enclosing Ball

7.2.1 Definition

An object of the class Min sphere d<Traits> is the unique sphere of

smallest volume enclosing a finite (multi)set of points in d-dimensional
Euclidean space Ed . For a set P we denote by ms(P ) the smallest sphere
that contains all points of P . Note that ms(P ) can be degenerate,
i.e. ms(P ) = ∅ if P = ∅ and ms(P ) = {p} if P = {p}.

An inclusion-minimal subset S of P with ms(S) = ms(P ) is called a

support set, the points in S are the support points. A support set has
size at most d+1, and all its points lie on the boundary of ms(P ). In
general, neither the support set nor its size are unique.

The underlying algorithm can cope with all kinds of input, e.g. P may
be empty or points may occur more than once. The algorithm computes
a support set S together with the center and the (squared) radius of the
smallest enclosing sphere.

7.2.2 Solution

Given the point set as P = { p1 , . . . , pn }, let x∗ = (x∗1 , . . . , x∗n ) be an

optimal solution to
Pn
(MB) minimize xT C T C x − i=1 pTi pi xi
Pn
subject to i=1 xi = 1 (7.3)
x ≥ 0,
112 Chapter 7. Geometric Optimization Problems in Cgal

with C = (p1 , . . . , pn ). Then the support set is determined by the posi-

tive x∗i s, namely

S = {pi ∈ P | x∗i > 0} .

7.2.3 Implementation

Similar to the polytope distance problem, the quadratic program (7.3)

can be represented as follows. Matrix A contains only a single row of
ones and b is the one-dimensional 1-vector. The entries of c are the
input points’ negated squared lengths. They are computed in advance
and stored in a vector of the points’ coordinate type. Again, the implicit
representation of D is the most interesting part. Here, we can reuse
the two iterator classes described in the previous section. The only
difference is that C contains only the points of P .

The smallest enclosing ball problem has the special properties of a sym-
metric objective matrix and no inequality constraints. The following
class template collects all types needed by the QPE to solve (7.3).

template < class Traits, class ET_ >

class MB_rep {
typedef ... Point; // point type (from Traits)
typedef ... CT; // points’ coordinate type
public:
typedef Const_value_iterator< Const_value_iterator<CT> >
A_iterator;
typedef Const_value_iterator< CT > B_iterator;
typedef std::vector< CT >::const_iterator C_iterator;
typedef PD_D_iterator< Point > D_iterator;

enum Row_type { EQUAL };

typedef Const_value_iterator< Row_type > R_iterator;

typedef Tag_false Is_linear;

typedef Tag_true Is_symmetric;
typedef Tag_true Has_no_inequalites;

typedef ET_ ET;

};
7.3. Smallest Enclosing Annulus 113

7.3 Smallest Enclosing Annulus

7.3.1 Definition

An object of the class Min annulus d<Traits> is the unique annulus

(region between two concentric spheres with radii r and R, r ≤ R) enclo-
sing a finite (multi)set of points in d-dimensional Euclidean space Ed ,
where the difference R2 − r2 is minimal. For a point set P we denote
by ma(P ) the smallest annulus that contains all points of P . Note that
ma(P ) can be degenerate, i.e. ma(P ) = ∅ if P = ∅ and ma(P ) = {p}
if P = {p}.

An inclusion-minimal subset S of P with ma(S) = ma(P ) is called a

support set, the points in S are the support points. A support set has
size at most d+2, and all its points lie on the boundary of ma(P ). In
general, the support set is not necessarily unique.

The underlying algorithm can cope with all kinds of input, e.g. P may
be empty or points may occur more than once. The algorithm computes
a support set S together with the center and the (squared) radii of the
smallest enclosing annulus.

7.3.2 Solution

Given the point set as P = { p1 , . . . , pn }, consider an optimal solution

to the linear program
iT i iT i
Pn Pn
(MA’) minimize i=1 p p µi − i=1 p p λi
Pn n
2pij λi + i=1 2pij µi = 0
P
subject to j = 1...d
Pi=1
n
λi = 1 (7.4)
Pi=1
n
i=1 µi = 1
λ, µ ≥ 0 .

Then the values of the dual variables determine the solution to the
smallest enclosing annulus problem. Let A−1 B be the lower left part of
the final basis inverse (cf. Section 6.3.1). The center c∗ is obtained by

(α∗ , β ∗ , c∗1 , . . . , c∗d ) = −cTB A−1

B
114 Chapter 7. Geometric Optimization Problems in Cgal

while the squared radii are computed from α∗ and β ∗ as

r2 = α∗ + ||c∗ ||2 ,
R2 = β ∗ + ||c∗ ||2 .

7.3.3 Implementation

The representation for (7.4) is straight forward. We keep A, b, and c

explicitly as (a vector of) vectors of the points’ coordinate type. All
entries are computed and stored in advance.

The dual version of the smallest enclosing annulus problem we use here
has the special properties of a linear objective function and no inequality
constraints. The following class template collects all types needed by
the QPE to solve (7.4).

template < class Traits, class ET_ >

class MA_rep {
typedef ... Point; // point type (from Traits)
typedef ... CT; // points’ coordinate type
public:
typedef std::vector< std::vector<CT> >::const_iterator
A_iterator;
typedef std::vector< CT > B_iterator;
typedef std::vector< CT > C_iterator;
typedef CT** D_iterator; // not used

enum Row_type { EQUAL };

typedef Const_value_iterator< Row_type > R_iterator;

typedef Tag_true Is_linear;

typedef Tag_false Is_symmetric;
typedef Tag_true Has_no_inequalites;

typedef ET_ ET;

};

An implementation of the primal version of the smallest enclosing annu-

lus problem has been done for the experiments described in the next
chapter. It will become part of Cgal in a future release.
7.4. Conclusion 115

7.4 Conclusion

We sketched the implementations of three geometric quadratic program-

ming problems as part of Cgal. These are the primal version of the
polytope distance problem, the smallest enclosing ball problem, and the
dual version of the smallest enclosing annulus problem.

We showed how to represent the programs efficiently, especially the

implicit representation of the dense objective matrix D in the polytope
distance and the smallest enclosing ball problems has been discussed.

Implementations of the dual version of the polytope distance problem

and the primal version of the smallest enclosing annulus problem are
already done and will be available in a future release of Cgal.
Chapter 8

Experimental Results

We tested our solver on three different problems, namely polytope

distance, smallest enclosing ball, and smallest enclosing annulus, with
various settings. In addition, we tested our implementation (within
Cgal) of the smallest enclosing ellipse problem, which uses Welzl’s
method combined with our exact primitives as described in Chapter 4.

8.1 Test Environment

As our major test problem, we chose the smallest enclosing ball problem,
because there the largest number of competing implementations was
available to us. We performed extensive tests, and compared the results
to the ones obtained using the other codes (including the QP solver of
CPLEX).

The smallest enclosing annulus problem is mainly selected as a test

problem, because it is an LP problem, and benchmarks have already
been obtained for an exact LP solver on that problem. It turns out that
although we solve LP as a special case of QP, our method is faster than
the dedicated LP code described in [39]. This is due to the fact that
118 Chapter 8. Experimental Results

great care has been taken to ensure runtime efficiency (see last but one
chapter).

All tables show the performance of different codes on data of different

dimensions, for different numbers of points, and different arithmetics
(averaged over several runs in each case1 ). For smallest enclosing ball,
the codes that were used are the double-only code Miniball of Gärt-
ner [40], the algorithms of Leda [68] and Cgal [19] (in the latter case
also the dedicated 2-dimensional version), our new QP based method,
and finally the QP-solver of CPLEX. For the smallest enclosing annu-
lus, the exact LP solver of Gärtner [39] was compared with our new
method.

Most tests have been performed with pseudo-random input, i.e. the
coordinates were chosen as the lower-order 24 bits of the pseudo-random
numbers generated by the function random from the C++ standard
library. It uses a non-linear additive feedback random number gen-
erator with a very large period, which is approximately 16 (231 −1).
Since random does not use a linear congruential generator, it is not sub-
ject to ‘non-random’ effects as described in [41], as far as we know. For
smallest enclosing ball, annulus, and ellipse, we further have tested with
degenerate inputs, sets of points lying exactly or almost on a circle or
sphere.

The three types of arithmetics used are double (floating point only),
filtered (a standard floating point filter approach in case of Leda,
and our hybrid scheme in case of the QP method), as well as exact,
denoting full multiple precision number arithmetic.

In case of the QP solver, two pricing strategies are available, full and
partial. Partial pricing is almost always faster than full pricing, which
is therefore not tabulated for all tests.

With one exception, runtimes have been measured on a notebook PC

with a Mobile Pentium III CPU running at 500 MHz under Linux-2.4,
using the GNU g++ compiler, version 2.95.2, at optimization level -O3.
Because CPLEX was available to us only on a SUN Ultra-60 worksta-
tion, the results in Table 8.9 refer to that machine.

1 In most cases, we averaged the runtimes over 100 runs. Only in a few cases,

where a single run needed several minutes to complete, we averaged over 10 runs.
8.2. Polytope Distance 119

random points
d 10,000 100,000
2 1.4 s 15.8 s
3 1.7 s 18.2 s
5 2.7 s 25.5 s
10 5.1 s 48.3 s
15 8.1 s 1:13 min
20 11.9 s 1:44 min
30 29.2 s 3:51 min
50 1:36 min 10:06 min

Table 8.1: Runtimes on random polytope distance problems

8.2 Polytope Distance

Table 8.1 shows the results of our solver (the best available setting: par-
tial filtered pricing) for polytope distance problems with various random
point sets in various dimensions. The main ‘message’ of the table is
that the solver is able to handle large instances fast and exact. A more
detailed picture is obtained from the results for smallest enclosing balls,
where we compete against other codes.

8.3 Smallest Enclosing Ball

While all methods that are considered can handle large point sets in
dimension 2, there are quite some differences in efficiency between float-
ing point, filtered and exact versions of the same code (Table 8.2).
Usually, our exact QP solver is only slightly slower than the double ver-
sions of the other codes, but dramatically faster than their exact imple-
mentations. The Leda version is still comparable, because it applies
a filtered approach itself. In cases the table entry is blank, we did not
wait for the result, the time here is at least half an hour.

As degenerate inputs for d = 2, we have chosen two sets of points lying

exactly on a circle. The small set has coordinates such that the squares
still fit into a double value, while the large one has not. The only double
implementations that can reasonably handle those sets are the Miniball
routine, and Cgal’s dedicated 2-dimensional code (Table 8.3). Again,
120 Chapter 8. Experimental Results

random points
Algorithm (d = 2)
10,000 100,000 1,000,000
Miniball (double) 4 ms 50 ms 646 ms
LEDA Min circle (double) 14 ms 223 ms 2.8 s
CGAL Min sphere (double) 13 ms 169 ms 1.7 s
CGAL Min circle (double) 45 ms 487 ms 4.6 s
QP Solver (partial,double) 14 ms 160 ms 1.6 s
QP Solver (full,double) 28 ms 331 ms 3.4 s
LEDA Min circle (filtered) 51 ms 561 ms 6.2 s
QP Solver (partial,filtered) 31 ms 327 ms 3.2 s
QP Solver (full,filtered) 44 ms 498 ms 5.5 s
QP Solver (partial,exact) 1.7 s 16.9 s 2:49 min
QP Solver (full,exact) 3.3 s 31.8 s
CGAL Min sphere (exact) 3.2 s 30.4 s
CGAL Min circle (exact) 5.3 s 52.2 s

Table 8.2: Runtimes on random miniball problems, d = 2

points on circle
Algorithm (d = 2)
6,144 13,824 6,144 (perturbed)
Miniball (double) <1 ms 5 ms 5 ms
CGAL Min circle (double) 10 ms 29 ms
LEDA Min circle (filtered) 1.27 s 3.1 s 6.5 s
QP Solver (partial,filtered) 412 ms 2.5 s 60 ms
QP Solver (full,filtered) 431 ms 4.8 s 110 ms
QP Solver (partial,exact) 416 ms 7.1 s 1.3 s
QP Solver (full,exact) 845 ms 23.7 s 4.3 s
CGAL Min sphere (exact) 423 ms 1.0 s 3.4 s
CGAL Min circle (exact) 894 ms 1.9 s 3.4 s

Table 8.3: Runtimes on degenerate miniball problems, d = 2

points on sphere
Algorithm (d = 3)
10,000 (perturbed)
Miniball (double) 19 ms
QP Solver (partial,filtered) 123 ms
QP Solver (full,filtered) 127 ms
QP Solver (partial,exact) 10.9 s
QP Solver (full,exact) 11.6 s
CGAL Min sphere (exact) 23.8 s

Table 8.4: Runtimes on degenerate miniball problem, d = 3

8.3. Smallest Enclosing Ball 121

the exact QP solver is faster than Leda’s filtered approach, and much
faster than all exact versions. An interesting phenomenon occurs when
one slightly perturbs the points, so that they are no longer cocircular.
The QP solver becomes much faster because the perturbation already
suffices to make the built-in error bounds work effectively: to verify
optimality in the last iteration, no exact checks are necessary anymore,
while they extensively happen for the non-perturbed degenerate input.
In d = 3, we have chosen a point set almost on a sphere, obtained by
tiling the sphere according to longitude and latitude values. The only
double code still able to handle this problem is Miniball; our filtered
approach, however, is only by a factor of six slower, while the exact
versions are out of the game (Table 8.4).
We also have results for higher dimensions (Tables 8.5, 8.6, and 8.7).
These show how the missing ‘curse of dimensionality’ in the QP solver
compensates for the effect of exact arithmetic, so that our exact solver
is already faster than the inexact solver Miniball in dimension 20, for
10,000 points. (For n = 100,000 we are not far off, but for 1,000,000
points we reach the machine’s memory limit). Note that the table entry
is blank for most pure double versions in higher dimensions, which
means that the results were too inaccurate.
Probably most interesting is that problems up to dimension 100 can
routinely be handled, and even for 10,000 points in d = 200, we only
need about three and a half minutes (Table 8.8). It should be noted
that these results hold for random points, where we observed that the
number of points that determine the final ball is quite small (much
smaller than the dimension itself). In this situation, the QP-bases the
algorithm needs to handle are relatively small, which makes the exact
arithmetic fast.
Finally, we have performed comparisons with the QP solver of CPLEX
(an interior point code) using formulation (3.7) on page 41, showing
that such codes are not competitive in our scenario (Table 8.9). It is
interesting to note that the performance of the CPLEX solver mainly
depends on the product of n and d (which is closely related to the
number of nonzeros in the problem description), while we pay a penalty
for larger values of d. However, the results show that our exact method
is still superior to CPLEX for dimensions below 30, which is the range
of dimensions our code is made for. It is clear that CPLEX’s method
will become superior as d goes higher up.
 122
random points (10,000)
Algorithm
d 2 3 5 10 15 20
Miniball (double) 4 ms 7 ms 16 ms 57 ms 168 ms 1.0 s
CGAL Min sphere (double) 13 ms 18 ms
QP Solver (partial,double) 14 ms 17 ms 26 ms 62 ms
QP Solver (partial,filtered) 31 ms 41 ms 74 ms 219 ms 408 ms 847 ms
QP Solver (partial,exact) 1.7 s 2.1 s 3.4 s 7.2 s 11.5 s 17.1 s
CGAL Min sphere (exact) 3.2 s 6.1 s 15.7 s 1:07 min 3:40 min

Table 8.5: Runtimes on random miniball problems, n = 10,000

random points (100,000)

Algorithm
d 2 3 5 10 15 20
Miniball (double) 50 ms 102 ms 249 ms 792 ms 1.5 s 3.2 s
CGAL Min sphere (double) 169 ms 256 ms 546 ms
QP Solver (partial,double) 160 ms 183 ms
QP Solver (partial,filtered) 327 ms 403 ms 644 ms 1.4 s 2.5 s 5.9 s
QP Solver (partial,exact) 16.9 s 20.8 s 52.4 s 1:02 min 1:33 min 2:15 min
CGAL Min sphere (exact) 30.4 s 1:01 min 2:50 min

Table 8.6: Runtimes on random miniball problems, n = 100,000

Chapter 8. Experimental Results
8.4. Smallest Enclosing Annulus 123

random points (1,000,000)

Algorithm
d 2 3 5 10 15
Miniball (double) 646 ms 1.1 s 2.5 s 9.2 s 7.3 s
CGAL Min sphere (double) 1.7 s 2.6 s 4.8 s 15.6 s
QP Solver (partial,double) 1.6 s 1.8 s
QP Solver (partial,filtered) 3.2 s 3.8 s 5.6 s 14.0 s 23.4 s
QP Solver (partial,exact) 2:49 min

Table 8.7: Runtimes on random miniball problems, n = 1, 000,000

random points random points

d 10,000 100,000 d 10,000 100,000
30 3.3 s 13.1 s 30 42 48
50 10.0 s 53.3 s 50 53 61
100 25.2 s 1:32 min 100 55 70
200 3:23 min 200 81
300 10:18 min 300 114

Table 8.8: Runtimes and number of iterations on random miniball

problems, QP solver with partial filtered pricing

random points
Algorithm
100,000 (d = 3) 10,000 (d = 30)
CPLEX (double) 12.8 s 10.2 s
QP Solver (partial,filtered) 671 ms 6.8 s

Table 8.9: Runtimes on two random miniball problems, compared to

CPLEX

8.4 Smallest Enclosing Annulus

We have tested our QP solver against the exact solver described in [39]
which employs basically the same combination of exact and floating
point arithmetic (denoted as LP simplex in the tables). However, as
the results show, our solver is even faster than the dedicated LP solver.
This is due to the fact that we invested much effort in optimizing the
code.
124 Chapter 8. Experimental Results

random points (d = 2)
Algorithm
10,000 100,000
QP Solver (partial,double) 7 ms 115 ms
QP Solver (full,double) 31 ms 416 ms
LEDA Min annulus (double) 2.2 s 28.7 s
QP Solver (partial,filtered) 20 ms 192 ms
QP Solver (full,filtered) 41 ms 546 ms
LP Simplex (partial,filtered) 246 ms 2.4 s
LEDA Min annulus (filtered) 19.6 s 3:52 min
QP Solver (partial,exact) 4.0 s 48.1 s
QP Solver (full,exact) 13.8 s 2:20 min

Table 8.10: Runtimes on random annulus problems, d = 2

points on circle (d = 2)
Algorithm
6,144 13,824 6,144 (perturbed)
QP Solver (partial,filtered) 0.8 s 2.1 s 24 ms
QP Solver (full,filtered) 0.8 s 2.1 s 38 ms
LP Simplex (partial,filtered) 0.9 s 1.7 s 159 ms
LEDA Min annulus (filtered) 2.8 s 6.5 s
QP Solver (partial,exact) 0.8 s 2.1 s 3.3 s
QP Solver (full,exact) 4.1 s 13.5 s 14.4 s

Table 8.11: Runtimes on degenerate annulus problems, d = 2

points on sphere (d = 3)
Algorithm
10,000 (perturbed)
QP Solver (partial,double) 39 ms
QP Solver (full,double) 58 ms
QP Solver (partial,filtered) 88 ms
QP Solver (full,filtered) 102 ms
LP Simplex (partial,filtered) 311 ms
QP Solver (partial,exact) 18.2 s
QP Solver (full,exact) 29.5 s

Table 8.12: Runtimes on degenerate annulus problem, d = 3

 random points (10,000)
Algorithm
d 2 3 5 10 15 20 30
QP Solver (partial,double) 7 ms 15 ms 20 ms 64 ms 143 ms 337 ms
QP Solver (full,double) 31 ms 76 ms 135 ms 546 ms 1.0 s 2.0 s
QP Solver (partial,filtered) 20 ms 44 ms 101 ms 795 ms 4.1 s 14.2 s 1:31 min
QP Solver (full,filtered) 41 ms 92 ms 182 ms 957 ms 3.7 s 11.7 s
LP Simplex (partial,filtered) 246 ms 273 ms 359 ms 1.3 s 4.9 s 16.5 s 1:33 min
QP Solver (partial,exact) 4.0 s 5.8 s 10.3 s 40 s 2:11 min 5:12 min
QP Solver (full,exact) 13.8 s 26.7 s 55.7 s 4:25 min 13:36 min
Table 8.13: Runtimes on random annulus problems, n = 10,000
8.4. Smallest Enclosing Annulus

random points (100,000)

Algorithm
d 2 3 5 10 15 20 30
QP Solver (partial,double) 115 ms 118 ms 164 ms 340 ms 747 ms 1.7 s
QP Solver (full,double) 466 ms 748 ms 1.6 s 4.9 s
QP Solver (partial,filtered) 192 ms 214 ms 350 ms 1.2 s 5.3 s 18.2 s 1:52 min
QP Solver (full,filtered) 547 ms 841 ms 1.7 s 6.1 s
LP Simplex (partial,filtered) 1.1 s 2.4 s 2.6 s 4.0 s 8.8 s 23.7 s 4:23 min
QP Solver (partial,exact) 49.1 s 53.9 s 1:28 min 4:04 min
QP Solver (full,exact) 2:22 min 5:43 min 11:25 min
Table 8.14: Runtimes on random annulus problems, n = 100,000
125
126 Chapter 8. Experimental Results

As before, the filtered approach is much faster than the exact approach,
and comparable to pure floating point solutions. Unlike in the case
of smallest enclosing ball, double versions are still able to compute the
correct result in higher dimensions, which indicates that true QP is more
challenging for the numerics than LP (Tables 8.10, 8.13, and 8.14). We
used the dual version (3.9) on page 43 for our tests, which turned out
to be always faster than the primal version (3.8) on page 42. The more
involved pivot step of the primal version (due to the handling of the
slack variables) and a higher number of iterations resulted in a slow
down by a factor of at least two.

In case of degenerate input (points on a circle), we observe the same

phenomenon as with smallest enclosing balls: as soon as we slightly
perturb the points, the filtered approach gets much faster, because less
exact computations are necessary (Tables 8.11 and 8.12).

8.5 Smallest Enclosing Ellipse

Table 8.15 shows the results of our Cgal implementation for smallest
enclosing ellipse problems. We tested the method on random point sets
of different size as well as on degenerate sets consisting of points lying
on or almost on a circle (described above). Note, Cgal Min circle
implements the same method of Welzl for the smallest enclosing circle
problem. A comparison of the runtimes here with those in Tables 8.2
and 8.3 shows the following. As expected, the much more involved

CGAL Min ellipse

Point Set (d = 2)
n (double) (exact)
10,000 72 ms 50.1 s
random 100,000 750 ms 8:51 min
1,000,000 7.3 s
6,144 4 ms 6.6 s
on circle
13,824 12 ms 14.0 s
on circle (perturbed) 6,144 15 ms 5.6 s

Table 8.15: Runtimes on smallest enclosing ellipse problems, d = 2

8.6. Conclusion 127

arithmetic in the primitive operations in case of the smallest enclosing

ellipse causes a significant increase in runtimes.

8.6 Conclusion

We performed numerous tests of our solver on the quadratic program-

ming problems polytope distance, smallest enclosing ball, and smallest
enclosing annulus, with different types of input data.

Our major test problem, namely the smallest enclosing ball problem,
was used to compare our solver with other implementations. Among the
competitors were mathematical programming codes as well as dedicated
geometric algorithms.

Almost always the partial filtered pricing strategy was best. Using this
combination of exact and floating point arithmetic yields very good
runtimes. They are not far behind pure floating point implementations,
but much faster than codes using exact arithmetic for all numerical
operations.

For the smallest enclosing annulus problem, which is an LP problem,

we even outperformed a dedicated LP code. This is due to the fact that
great care has been taken to ensure runtime efficiency.

Finally, we showed that smallest enclosing ellipse problems with up to

100,000 points can be solved exactly using Welzl’s method combined
with our exact primitives.
 Part III

Applications
Chapter 9

Two Applications

This chapter presents two applications, where our method was used
to solve ‘real-world’ problems. As part of a joint project with a big
Swiss bank, our quadratic programming solver was used to solve a large
portfolio optimization problem. In another cooperation with a German
company developing medical navigation systems, the smallest enclosing
annulus implementation was used to calibrate the tip of medical probes.

9.1 Portfolio Optimization

In this section we describe the application of the quadratic programming

solver in asset management.

9.1.1 Application Background

Recently, a coherent measure of risk, the so called Conditional Value-

at-Risk, is gaining ground both in academia and in financial indus-
try [103, 84, 65]. An extension of this methodology for multi-period
portfolio optimization is desirable [23, 14]. In this context, a diploma
thesis’ project was assigned to two students at the Institute for Opera-
tions Research (IFOR), ETH Zurich. Their subject was to compare the
132 Chapter 9. Two Applications

risk measures ‘Mean-Variance and Conditional Value-at-Risk in Port-

folio Optimization’ [8]. The asset management division of Credit Suisse
(CSAM) in Zurich acted as industrial partner for this diploma thesis.

The models used in portfolio optimization are usually formulated as

linear or quadratic programs. One of the tasks in the project was to
compare different solvers for linear and quadratic programming prob-
lems. A preliminary version of our exact implementation described in
Chapter 6 was one of them. During the project meetings, the need for
exact solutions was discussed. The issue was raised by CSAM, because
they encountered numerical problems when they tried to solve large
portfolio optimization problems.

9.1.2 Problem Description

CSAM provided us with one of their ‘problematic’ problems. It is

a multi-period portfolio optimization problem of a Swiss pension fonds
(‘Schweizer Pensionskasse’).

The model includes 11 financial instruments, i.e. stocks or investment

fonds, for which the best combination should be found. Transaction
costs for buying and selling instruments are also modelled, resulting in
33 variables for one time period. The model contains five periods, so
it has 165 variables in total. The problem is formulated as a quadratic
program by using the Markowitz model to controll the risk of the port-
folio.

At CSAM, they tried to solve this problem using CPLEX. It was neither
able to find a feasible solution nor to show infeasibility of the quadratic
program. CPLEX just terminated after some amount of time, indicat-
ing that it detected ‘cycling’. Further details, e.g. version and parameter
settings of CPLEX and the particular reason for its failure, were not
available to us.

9.1.3 Results

The quadratic program of CSAM was given in MPS format, so we reused

an MPS reader from a previous implementation of an inexact solver
9.2. Stylus Calibration 133

developed together with Bernd Gärtner. We extended the reader for

quadratic programs, i.e. we added the handling of a QMATRIX section.

The problem finally feeded into the solver consisted of 165 original and
211 slack variables, resulting in a total of 371 variables (the preliminary
version of the solver used here was not able to handle slack variables
implicitly). It contained 271 constraints, devided in 60 equality and
211 inequality constraints.

Our solver (using full exact pricing) needed an overall runtime of 1 hour
and 53 minutes. After 395 iterations in phase I, it returned that the
problem is INFEASIBLE. We checked the solution and found it to be
correct.

Since we were able to ‘solve’ the problem, our approach seems promis-
ing in this application domain. Although the computation took quite a
while, future versions using partial filtered pricing on up-to-date hard-
ware will need only a fraction of the observed runtime. At least we
showed that our exact solver might be used as a tool in portfolio opti-
mization.

9.2 Stylus Calibration

We describe the solution of a calibration problem that arised from

a medical application in neurosurgery.

9.2.1 Application Background

In recent years, intraoperative navigation systems were recognized as

a very powerful tool for surgical cases. On the one hand, they improve
the results of such operations, while on the other hand, they make surgi-
cal interventions yet possible that where considered inoperable before.
One such system is the computer aided neurosurgery navigation sys-
tem NEN [87] developed by Functional Imaging Technologies (FIT) in
Berlin, Germany.
134 Chapter 9. Two Applications

The general aim of these systems is to enable a surgeon to utilize pre-

operatively gathered information more effectively during the operation.
This information is usually a three-dimensional image set of the region
of interest, obtained by a computed tomography (CT) or magnetic reso-
nance (MR) scanner. The overall idea consists in linking the information
from the image set directly to the operating environment for navigation
purposes. Like in most such guidance systems (see for example [33] and
the references there) the basic approach to solve this question relies on a
technique called image registration. A set of markers (so called fiducials)
is attached to the skin (or even bone implanted in other systems) and
special points on the markers serve as orientation landmarks. These
landmarks are visible in the pre-operative volumetric image set and
define a set of 3D ‘image’ points. Then, immediately prior to the inter-
vention, a 3D tracking system is used to localize the landmark points
a second time in the operating environment while the patient’s head is
fixed. This defines another set of 3D ‘world’ points. The image registra-
tion process computes a transformation defined by the two point sets,
that maps each point from ‘world’ space to its counterpart in ‘image’
space.1

9.2.2 Problem Description

The 3D tracking system used by the NEN navigation system is based on

electro-magnetic fields. A transmitter generates a pulsed DC magnetic
field, which allows a corresponding receiver to localize itself within the
electro-magnetic field. The position and orientation of the receiver in
3-space is measured at regular intervals and the information is handed
over to the navigation system.

The surgeon uses a medical probe, the so called stylus, to navigate in

the operating environment, see Figure 9.1. The receiver of the tracking
system is integrated in the handle of the stylus. Since we are interested
in the position (and orientation) of the stylus’ tip, we have to calibrate
the stylus, i.e. the offset vector between the receiver in the handle and
the tip of the stylus has to be determined.

1 Actually, the cooperation with FIT started with a project on the image regis-

tration problem, see [54].

9.2. Stylus Calibration 135

Figure 9.1: Stylus of the navigation system

9.2.3 Solution

The idea of the calibration algorithm is as follows. We place the tip of

the stylus in a special tool, such that the stylus can be moved and ro-
tated but the tip’s position remains fixed, see Figure 9.2. When moving
the stylus, all measured positions of the receiver lie (theoretically) on
a particular half-sphere in 3-space. The tip’s position is the center of
the smallest enclosing annulus of the set P of measured positions. Note
that the smallest enclosing ball of P cannot be used here, because the
acquired positions cover only a half-sphere. In general, the center of this
half-sphere does not coincide with the center of the smallest enclosing
ball of P .

In practice, the points in P are distorted due to measurement errors.

There might also be outliers due to extraneous causes (e.g. through
metallic objects) that distort the electro-magnetic field. We use the
following heuristic to cope with these problems. After computing the
smallest enclosing annulus of P , we check its width, i.e. the difference
of R and r, cf. Section 3.3. If the annulus is wider than a predefined
bound, we remove points from P that are ‘most extreme’ with respect
to the current annulus. By ‘most extreme’ we mean those points, whose
distance to the annulus’ center deviates most from the average distance
to the center of all points in P . Then the smallest enclosing annulus
is recomputed. This step is repeated, until the width of the resulting
annulus is smaller than the predefined bound. We end up with the fol-
lowing algorithm (ma(P ) denotes the smallest enclosing annulus of P ):
136 Chapter 9. Two Applications

Figure 9.2: Calibration setup

Algorithm 9.1 (returns the position of the stylus’ tip)

StylusCalibration(P ):
a := ma(P )
WHILE width(a) > bound DO
P := P \ {p ∈ P | p is a ‘most extreme’ outlier}
a := ma(P )
END
RETURN center(a)

9.2.4 Experimental Results

The navigation system has two kind of styli. The difference lies in
the length of the pointer, resulting in different length of the offset vec-
tors. The short version has an offset of about 12cm, the long version of
about 19cm.

For each stylus to calibrate, we acquired between 5,000 and 10,000 posi-
tion samples. Usually one or two iterations in Algorithm 9.1 were suf-
ficient to obtain an annulus of width smaller than 2mm (short version)
and 3mm (long version). The annulus’ center was then used to compute
the offset vector for each point in the final set P . Averaging over all
these vectors gave the desired offset vector for the stylus.
9.2. Stylus Calibration 137

In a postprocessing step, we checked the quality of the calibration. For

each point in the final P , the position of the tip was computed using the
now available offset vector of the stylus. These positions were compared
to the ‘real’ position of the tip, namely the center of the final annulus.
The maximal deviations of 1.2mm for the short styli, and 1.8mm for
the long version, were perfectly satisfying for this application.
Bibliography

[1] I. Adler and R. Shamir. A randomized scheme for speeding up algo-

rithms for linear and convex programming with high constraints-
to-variable ratio. Math. Programming, 61:39–52, 1993.
[2] P. K. Agarwal, B. Aronov, S. Har-Peled, and M. Sharir. Approx-
imation and exact algorithms for minimum-width annuli and shells.
Discrete Computational Geometry, 24(4):687–705, 2000.
[3] N. Amenta. Computational geometry software. In Handbook of
Discrete and Computational Geometry, pages 951–960. CRC Press,
1997.
[4] J. Arvo, editor. Graphics Gems II. The graphics gems series.
Academic Press, 1991.
[5] D. Avis. A C implementation of the reverse search vertex enumer-
ation algorithm. ftp://mutt.cs.mcgill.ca/pub/C.
[6] F. Avnaim. C++GAL: A C++ Library for Geometric Algorithms.
INRIA Sophia-Antipolis, 1994.
[7] J. E. Baker, R. Tamassia, and L. Vismara. GeomLib: Algorithm
engineering for a geometric computing library, 1997. (Preliminary
report).
[8] S. Barbey and P. Schiltknecht. Portfolio Optimierung mit Marko-
witz und Conditional Value-at-Risk. Diploma thesis, ETH Zurich,
2001.
[9] V. Barnett. The ordering of multivariate data. J. Roy. Statist.
Soc., 139:318–354, 1976.
140 Bibliography

[10] J. J. Barton and L. R. Nackman. Scientific and Engineering C++.

Addison-Wesley, 1997.
[11] D. P. Bertsekas. Nonlinear Programming. Athena Scientific, Bel-
mont, Massachusetts, 1995.
[12] R. E. Bixby, J. W. Gregory, I. J. Lustig, R. E. Marsten, and D. F.
Shanno. Very large-scale linear programming: a case study in com-
bining interior point and simplex methods. Operations Research,
40(5):885–897, 1992.
[13] G. Booch and M. Vilot. Simplifying the Booch components. In
S. Lippman, editor, C++ Gems, pages 59–89. SIGS publications,
1996.
[14] A. Boriçi and H.-J. Lüthi. Multi-period portfolio optimization by
conditional value-at-risk: a proposal. (preprint), 2001.
[15] C. Bouville. Bounding ellipsoids for ray-fractal intersection. In
SIGGRAPH, pages 45–52, 1985.
[16] H. Brönnimann, C. Burnikel, and S. Pion. Interval arithmetic
yields efficient dynamic filters for computational geometry. Dis-
crete Applied Mathematics, 109(1–2):25–47, 2001.
[17] H. Brönnimann, I. Emiris, V. Pan, and S. Pion. Computing exact
geometric predicates using modular arithmetic with single preci-
sion. In Proc. 13th Annu. ACM Sympos. Comput. Geom., pages
174–182, 1997.
[18] International standard ISO/IEC 14882: Programming languages –
C++. American National Standards Institute, 11 West 42nd Street,
New York 10036, 1998.
[19] CGAL Reference Manual, 2001. Ver. 2.3. http://www.cgal.org/.
[20] V. Chvátal. Linear Programming. W. H. Freeman, New York, NY,
1983.
[21] K. L. Clarkson. A Las Vegas algorithm for linear programming
when the dimension is small. J. ACM, 42(2):488–499, 1995.
[22] L. Danzer, D. Laugwitz, and H. Lenz. Über das Löwnersche Ellip-
soid und sein Analogon unter den einem Eikörper eingeschriebenen
Ellipsoiden. Arch. Math., 8:214–219, 1957.
Bibliography 141

[23] A. C. Davison, C. G. Diderich, and V. Nicoud. Analytical multi-

period portfolio optimization. (preprint), February 2000.
[24] P. de Rezende and W. Jacometti. Geolab: an environment for
development of algorithms in computational geometry. In Proc.
5th Canad. Conf. Comput. Geom., pages 175–180, 1993.
[25] T. D. DeRose. Geometric programming: A coordinate-free
approach. In Theory and Practice of Geometric Modeling, pages
291–303, Blaubeuren, FRG (Oct 1988), 1989. Springer-Verlag.
[26] M. E. Dyer. A class of convex programs with applications to com-
putational geometry. In Proc. 8th Annu. ACM Symp. on Compu-
tational Geometry, pages 9–15, 1992.
[27] B. C. Eaves and R. M. Freund. Optimal scaling of balls and poly-
hedra. Math. Programming, 23:138–147, 1982.
[28] J. Edmonds and J. Maurras. Note sur les Q-matrices d’Edmonds.
Rech. Opér (RAIRO), 31(2):203–209, 1997.
[29] P. Epstein, J. Kavanagh, A. Knight, J. May, T. Nguyen, and J.-
R. Sack. A workbench for computational geometry. Algorithmica,
11:404–428, 1994.
[30] A. Fabri, G.-J. Giezeman, L. Kettner, S. Schirra, and S. Schönherr.
The Cgal kernel: A basis for geometric computation. In M. C. Lin
and D. Manocha, editors, Proc. ACM Workshop on Applied Com-
putational Geometry, volume 1148 of Lecture Notes in Computer
Science, pages 191–202. Springer-Verlag, 1996.
[31] A. Fabri, G.-J. Giezeman, L. Kettner, S. Schirra, and S. Schönherr.
On the design of Cgal, a computational geometry algorithms
library. Software – Practice and Experience, 30:1167–1202, 2000.
[32] G. Fischer. Lineare Algebra – Eine Einführung für Studien-
anfänger. 12., verbesserte Auflage. Vieweg, Braunschweig, Ger-
many, 2000.
[33] J. M. Fitzpatrick, J. B. West, and C. R. Maurer. Predicting error in
rigid-body point-based registration. IEEE Transactions on Medical
Imaging, 17(5):694–702, 1998.
[34] G. Forsythe and C. Moler. Computer Solutions of Linear Algebra
Systems. Prentice-Hall, Englewood Cliffs, NJ, 1967.
142 Bibliography

[35] R. M. Freund and J. B. Orlin. On the complexity of four polyhedral

set containment problems. Math. Programming, 33:139–145, 1985.
[36] K. Fukuda. cdd+ reference manual.
http://www.ifor.math.ethz.ch/~fukuda/cdd home/cdd.html.
[37] B. Gärtner. Randomized Optimization by Simplex-Type Methods.
PhD thesis, Freie Universität Berlin, 1995.
[38] B. Gärtner. Geometric optimization. Equinoctial School on
Geometry and Computation, ETH Zurich, 1997.
[39] B. Gärtner. Exact arithmetic at low cost – a case study in lin-
ear programming. Computational Geometry - Theory and Appli-
cations, 13:121–139, 1999.
[40] B. Gärtner. Fast and robust smallest enclosing balls. In Proc. 7th
Annu. European Sympos. Algorithms, volume 1643 of Lecture Notes
in Computer Science, pages 325–338. Springer-Verlag, 1999.
[41] B. Gärtner. Pitfalls in computing with pseudorandom determi-
nants. In Proc. 16th Annu. ACM Sympos. Comput. Geom., pages
148–155, 2000.
[42] B. Gärtner and S. Schönherr. Exact primitives for smallest enclo-
sing ellipses. Information Processing Letters, 68(1):33–38, 1998.
[43] B. Gärtner and S. Schönherr. Smallest enclosing circles – an exact
and generic implementation in C++. Technical Report B 98-04,
Fachbereich Mathematik und Informatik, Freie Universität Berlin,
1998.
[44] B. Gärtner and S. Schönherr. Smallest enclosing ellipses – an exact
and generic implementation in C++. Technical Report B 98-05,
Fachbereich Mathematik und Informatik, Freie Universität Berlin,
1998.
[45] B. Gärtner and S. Schönherr. An efficient, exact, and generic
quadratic programming solver for geometric optimization. In Proc.
16th Annu. ACM Sympos. Comput. Geom., pages 110–118, 2000.
[46] B. Gärtner and E. Welzl. Linear programming – randomization and
abstract frameworks. In Proc. 13th Annu. Symp. on Theoretical
Aspects of Computer Science (STACS), volume 1046 of Lecture
Notes in Computer Science, pages 669–687. Springer-Verlag, 1996.
Bibliography 143

[47] B. Gärtner and E. Welzl. A simple sampling lemma: Analysis and

applications in geometric optimization. Discrete Computational
Geometry, 25:569–590, 2001.
[48] G.-J. Giezeman. PlaGeo, a Library for Planar Geometry and Spa-
Geo, a Library for Spatial Geometry. Utrecht University, 1994.
[49] A. S. Glassner, editor. Graphics Gems. The graphics gems series.
Academic Press, 1990.
[50] R. N. Goldman. Illicit expressions in vector algebra. ACM Trans-
action on Graphics, 4(3):223–243, 1985.
[51] T. Granlund. GNU MP, The GNU Multiple Precision Arithmetic
Library, October 2000. Version 3.1.1. http://swox.com/gmp/.
[52] P. S. Heckbert, editor. Graphics Gems IV. The graphics gems
series. Academic Press, 1994.
[53] S. Hert, M. Hoffmann, L. Kettner, S. Pion, and M. Seel. An adapt-
able and extensible geometry kernel. In Proc. 5th Workshop on
Algorithms Engineering, volume 2141 of Lecture Notes in Com-
puter Science, pages 76–91. Springer-Verlag, 2001.
[54] F. Hoffmann, K. Kriegel, S. Schönherr, and C. Wenk. A simple and
robust geometric algorithm for landmark registration in computer
assisted neurosurgery. Technical Report B 99-21, Freie Universität
Berlin, Fachbereich Mathematik und Informatik, 1999.
[55] IEEE Computer Society. IEEE Standard for Binary Floating-Point
Arithmetic, IEEE Std 754, 1995. http://standards.ieee.org/.
[56] F. John. Extremum problems with inequalities as subsidiary con-
ditions. In Studies and Essays presented to R. Courant on his 60th
Birthday, pages 187–204. Interscience Publishers, NY, 1948.
[57] T. Keffer. The design and architecture of tools.h++. In S. Lippman,
editor, C++ Gems, pages 43–57. SIGS publications, 1996.
[58] L. Kettner. Software Design in Computational Geometry and
Contour-Edge Based Polyhedron Visualization. PhD thesis, ETH
Zurich, 1999. Diss. ETH No. 13325.
[59] D. Kirk, editor. Graphics Gems III. The graphics gems series.
Academic Press, 1992.
144 Bibliography

[60] R. Kumar and D. Sivakumar. Roundness estimation via random

sampling. In Proc. 10th ACM-SIAM Sympos. Discrete Algorithms,
pages 603–612, 1999.
[61] J. Lakos. Large Scale C++ Software Design. Addison-Wesley, 1996.
[62] K. Leichtweiß. Über die affine Exzentrizität konvexer Körper.
Arch. Math., 10:187–199, 1959.
[63] S. B. Lippman. Inside the C++ Object Model. Addison-Wesley,
1996.
[64] J. Matoušek, M. Sharir, and E. Welzl. A subexponential bound for
linear programming. Algorithmica, 16:498–516, 1996.
[65] H. Mausser and D. Rosen. Managing risk with expected shortfall.
In S. P. Uryasev, editor, Probabilistic Constrained Optimization:
Methodology and Applications, pages 204–225. Kluwer Academic
Publishers, 2000.
[66] K. Mehlhorn and S. Näher. The implementation of geometric algo-
rithms. In 13th World Computer Congress IFIP94, volume 1, pages
223–231, North-Holland, Amsterdam, 1994. Elsevier Science B.V.
[67] K. Mehlhorn and S. Näher. The LEDA Platform for Combinatorial
and Geometric Computing. Cambridge University Press, 1999.
[68] K. Mehlhorn, S. Näher, M. Seel, and C. Uhrig. The LEDA User
Manual, 2001. Version 4.3.
[69] S. Meyers. Effective C++. Addison Wesley, 1992.
[70] S. Meyers. More Effective C++. Addison Wesley, 1996.
[71] K. G. Murty and Y. Fathi. A critical index algorithm for nearest
point problems on simplicial cones. Math. Programming, 23:206–
215, 1982.
[72] D. R. Musser and A. Saini. STL Tutorial and Reference Guide:
C++ Programming with the Standard Template Library. Addison-
Wesley, 1996.
[73] D. R. Musser and A. A. Stepanov. Generic programming. In 1st
Intl. Joint Conf. of ISSAC-88 and AAEC-6, volume 358 of Lecture
Notes in Computer Science, pages 13–25. Springer-Verlag, 1989.
Bibliography 145

[74] D. R. Musser and A. A. Stepanov. Algorithm-oriented generic

libraries. Software – Practice and Experience, 24(7):623–642, 1994.
[75] N. C. Myers. Traits: a new and useful template technique. C++
Report, 1995.
[76] Y. Nesterov and A. Nemirovskii. Interior–Point Polynomial Algo-
rithms in Convex Programming, volume 13 of SIAM studies in ap-
plied mathematics. Society of Industrial and Applied Mathematics,
1993.
[77] J. Nievergelt, P. Schorn, M. de Lorenzi, C. Ammann, and A. Brüng-
ger. XYZ: a project in experimental geometric computation.
In Computational Geometry — Methods, Algorithms and Appli-
cations: Proc. Internat. Workshop Comput. Geom., volume 553
of Lecture Notes in Computer Science, pages 171–186. Springer-
Verlag, 1991.
[78] M. H. Overmars. Designing the computational algorithms library
Cgal. In M. C. Lin and D. Manocha, editors, Proc. ACM Work-
shop on Applied Computational Geometry, volume 1148 of Lecture
Notes in Computer Science. Springer-Verlag, 1996.
[79] A. L. Peressini, F. E. Sullivan, and J. J. Uhl. The Mathematics
of Nonlinear Programming. Undergraduate Texts in Mathematics.
Springer-Verlag, 1988.
[80] M. J. Post. Computing minimum spanning ellipses. Technical
Report CS-82-16, Department of Computer Science, Brown Uni-
versity, USA, 1982.
[81] M. J. Post. Minimum spanning ellipsoids. In Proc. 16th Annu.
ACM Sympos. Theory Comput., pages 108–116, 1984.
[82] L. Pronzato and E. Walter. Minimum-volume ellipsoids containing
compact sets: Applications to parameter bounding. Automatica,
30(11):1731–1739, 1994.
[83] V. T. Rajan. Optimality of the delaunay triangulation in Rd . In
Proc. 7th Annu. ACM Sympos. Comput. Geom., pages 357–363,
1991.
[84] R. T. Rockafellar and S. Uryasev. Optimization of conditional
value-at-risk. Journal of Risk, 2(3):21–41, 2000.
146 Bibliography

[85] S. Schirra. A case study on the cost of geometric computing. In

Proc. ALENEX’99, volume 1619 of Lecture Notes in Computer Sci-
ence, pages 156–176. Springer-Verlag, 1999.

[86] S. Schirra. Precision and robustness issues in geometric computa-

tion. In Handbook on Computational Geometry. Elsevier Science
Publishers, Amsterdam, The Netherlands, 1999.

[87] A. Schleich, S. Jovanovic, B. Sedlmaier, U. Warschewske, F. Olt-

manns, S. Schönherr, W. Oganovsky, B. Ohnesorge, T. Hoell, and
H. Scherer. Electromagnetic ENT navigation: the NEN system. In
Proc. 4th Annu. Conf. International Society for Computer Aided
Surgery, 2000.

[88] S. Schönherr. Berechnung kleinster Ellipsoide um Punktemengen.

Diploma thesis, Freie Universität Berlin, Germany, 1994.

[89] S. Schönherr. A quadratic programming engine for geometric opti-

mization – part I: the solver. Technical Report 387, ETH Zurich,
Institute of Theoretical Computer Science, December 2002.

[90] S. Schönherr. A quadratic programming engine for geometric opti-

mization – part II: the basis inverse. Technical Report 388, ETH
Zurich, Institute of Theoretical Computer Science, December 2002.

[91] S. Schönherr. A quadratic programming engine for geometric opti-

mization – part III: pricing strategies. Technical Report 389, ETH
Zurich, Institute of Theoretical Computer Science, December 2002.

[92] P. Schorn. Implementing the XYZ GeoBench: a programming en-

vironment for geometric algorithms. In Computational Geometry
— Methods, Algorithms and Applications: Proc. Internat. Work-
shop Comput. Geom., volume 553, pages 187–202, 1991.

[93] R. Seidel. Personal communication, 1997.

[94] K. Sekitani and Y. Yamamoto. A recursive algorithm for finding

the minimum norm point in a polytope and a pair of closest points
in two polytopes. Math. Programming, 61(2):233–249, 1993.

[95] N. Z. Shor and O. A. Berezovski. New algorithms for constructing

optimal circumscribed and inscribed ellipsoids. Opt. Methods and
Software, 1:283–299, 1992.
Bibliography 147

[96] Silicon Graphics Computer Systems. Standard Template Library

Programmer’s Guide, 1997. http://www.sgi.com/tech/stl/.

[97] B. W. Silverman and D. M. Titterington. Minimum covering

ellipses. SIAM J. Sci. Statist. Comput., 1:401–409, 1980.
[98] B. Spain. Analytical Conics. Pergamon Press, 1957.
[99] A. Stepanov and M. Lee. The Standard Template Library, October
1995. http://www.cs.rpi.edu/~musser/doc.ps.

[100] B. Stroustrup. The C++ Programming Language. Addison-Wesley,

special edition, 2000.
[101] D. M. Titterington. Optimal design: some geometrical aspects of
D-optimality. Biometrika, 62(2):313–320, 1975.

[102] D. M. Titterington. Estimation of correlation coefficients by ellip-

soidal trimming. Appl. Statist., 27(3):227–234, 1978.
[103] S. Uryasev. Conditional value-at-risk: Optimization algorithms
and applications. Financial Engineering News, 14, 2000.
[104] B. Wallis. Forms, vectors, and transforms. In A. S. Glassner,
editor, Graphics Gems, pages 533–538. Academic Press, 1990.
[105] E. Welzl. Smallest enclosing disks (balls and ellipsoids). In H. Mau-
rer, editor, New Results and New Trends in Computer Science,
volume 555 of Lecture Notes in Computer Science, pages 359–370.
Springer-Verlag, 1991.
[106] P. Wolfe. The simplex method for quadratic programming. Econo-
metrica, 27:382–398, 1959.
[107] P. Wolfe. Finding the nearest point in a polytope. Math. Program-
ming, 11:128–149, 1976.
Index

— A — of LP, 17
access to original problem, 83–84 of QP, 17, 21–22
active variable, 97, 102, 103, 104 basis inverse, 81, 82, 87–95, 96
algorithm implementation, 94–95
PartialFilteredPricing, rational representation,
104, 105 88–89, 94, 96
PartialPricing, 97 update, 89–94
SmEll, 48, 49, 50, 52, 59 basis matrix, 18, 24, 31, 87, 88
StylusCalibration, 136 boundary points, 47, 48, 49, 59
annulus bounding volume heuristic, 4
smallest enclosing, 4, 42–43,
113–114, 123–126, 135 — C —
application, 131–137
Cartesian (class), 71, 72, 78
financial, 131–133
Cartesian representation, 71–72,
medical, 133–137
73
asset management, 131
Cgal, iv, vi, vii, viii, ix, 2, 3, 9,
10, 59, 63–80, 81, 86,
— B — 105, 107, 111, 114, 115,
ball 117, 118, 119, 126
smallest enclosing, 4, 6, 7, basic library, 70, 74–79
39–41, 45, 83, 111–112, Cartesian representation,
119–123 71–72, 73
basic feasible solution, 17, 18, 21, circulator, 79
22 core library, 70
basic library, 3, 70, 74–79 geometric kernel, 69, 71–74
basic solution, 21–23 homogeneous representation,
basic variable, 17, 21, 22, 86, 89 71–72, 73
basis, 17, 18, 31, 88, 89, 96, 97 library structure, 69–70
150 Index

support library, 70 convex programming, 8, 45

traits class, 70, 76–79, 80, convex set, 47
107, 111, 113 convex hull points 2
circle (function), 78, 79
smallest enclosing, 3 Convex hull traits 2 (class),
class 78, 79
Cartesian, 71, 72, 78 CPLEX, 6, 132
Convex hull traits 2, 78, Cramer’s rule, 89
79
Homogeneous, 71, 72
Min annulus d, 113 — D —
Min sphere d, 111 D-optimal design, 8
Point 2, 71, 78 Dantzig’s rule, 97, 98, 99
Polygon 2, 78 degeneracy
Polytope distance d, 107 handling of -ies, 34–35
QPE basis inverse, 94 problem, 1–2
QPE full exact pricing, degenerate quadratic program,
103 86
QPE full filtered pricing, design goals, 81–82
103 dual version
QPE partial exact pricing, polytope distance, 38, 111
103 smallest enclosing annulus,
QPE partial filtered 42, 114, 126
pricing, 103
QPE pricing strategy, 101,
102 — E —
QPE solver, 82, 83, 85, 94, Ecg, 2
101 ellipse, 50, 51, 58
QPErep, 83, 84, 86 irrational, 57
Triangulation 2, 77 normalized, 51
collision detection, 4 smallest enclosing, 3, 50, 58,
compile time tag, 84–86, 94, 105 59, 126–127
computational geometry, 1, 2, 4, ellipsoid, 46
6 function, 46
software, 64–65 smallest enclosing, 7–8, 45,
Conditional Value-at-Risk, 131 46
conic, 51, 58 entering variable, 18, 86, 89, 90,
linear combination of -s, 51 95, 97, 98, 99, 102, 103,
normalized, 51 104
convex program, 46 error bound, 99
Index 151

exact arithmetic, 86, 98, 99, 102, hoping for the best, 7
104 hyperplane
exact number type, 86, 94 optimal separating, 5, 43
expecting the best and coping
with the worst, 7 — I —
expecting the worst, 7
image registration, 134
experimental results, 117–127
implementation
basis inverse, 94–95
— F — pricing strategy, 101–105
feasible solution, 16, 21, 22 implicit representation
filtered pricing, 98–101 of objective matrix, 83, 109,
financial application, 131–133 112
floating point filter, 86, 99, 104 in-ellipse test, 50–51
function |R| = 3, 52
convex hull points 2, 78, |R| = 4, 52–57
79 |R| = 5, 58
ellipsoid, 46 in-ellipsoid test, 50
objective, 16, 17, 18, 47, 88, inequality constraint, 110, 112
97 -, see also slack variable
handling of -s, 31–34
— G — inexact number type, 102
intraoperative navigation system,
Galia, 2, 63
133
generalized simplex method, 87
irrational ellipse, 57
generic programming, 65, 66–69,
iterator, 83, 84, 105
79, 81, 105
geometric kernel, 3, 69, 71–74
representation class, 72 — K —
geometric optimization, 3, 6, 7 Karush-Kuhn-Tucker conditions,
scenario, 5–6 20–21, 39
for QP, 20
— H — for UQP, 20
kernel, 3
handling of
geometric, 3
degeneracies, 34–35
KKT, see Karush-Kuhn-Tucker
inequality constraints, 31–34
conditions
heuristic
move-to-front, 50
Homogeneous (class), 71, 72 — L —
homogeneous representation, Lagrange multipliers, 20
71–72, 73 lazy evaluation, 105
152 Index

leaving variable, 18, 90, 103 numerical accuracy, 7

Leda, 64, 69, 70, 78, 118, 119,
121 — O —
leda real, 59
linear combination of conics, 51 object-oriented programming,
linear objective function, 82, 84, 65–66, 69, 80, 105
85 objective function, 16, 17, 18,
linear program, 15, 17, 81, 88, 47, 88, 97
89, 94, 98 linear, 82, 84, 85
linear programming, 3, 15 objective matrix, 82
basis, 17 implicit representation, 83,
LP, see linear programming 109, 112
LP-type problem, 4, 8 symmetric, 84, 85, 88, 110,
Löwner-John ellipsoid, 8 112
operation
primitive, 45, 50–58, 59
— M — operations research, 5
main results, 9 optimal containment problem, 4
medical application, 133–137 optimal separating hyperplane,
Min annulus d (class), 113 5, 43
Min sphere d (class), 111 optimal solution, 16, 21, 22
move-to-front heuristic, 50 optimization
multi-period portfolio geometric, 3
optimization, 131, 132 original variable, 89, 90

— N — — P —
navigation system partial filtered pricing, 103–105
intraoperative, 133 partial pricing, 97–98, 102
neurosurgery, 133 PartialFilteredPricing
nonbasic variable, 17, 18, 21, 22, (algorithm), 104, 105
89, 95, 96, 97, 98, 102, PartialPricing (algorithm),
103, 104 97
nondegenerate quadratic pivot rule, 96
program, 19 pivot step, 18, 23–31
normalized pricing, 24–25, 86
conic, 51 ratio test, 25–86
ellipse, 51 step 1, 25–27
number type, 82, 102 step 2, 27–28
exact, 86, 94 step 3, 28–30
inexact, 102 update, 31
Index 153

point linear, 15
boundary -s, 47, 48, 49, 59 object-oriented, 65–66, 69,
realizing -s, 108 80, 105
support -s, 47, 50, 107, 111, quadratic, 15, 35, 98, 105
113
Point 2 (class), 71, 78 — Q —
Polygon 2 (class), 78 Q-pivoting, 94
polytope distance, 3, 37–39, 83, QP, see quadratic programming
107–111, 112, 119 QP-basis, 21–22
dual version, 38, 111 size of, 22–23
primal version, 37, 110 QPE, see quadratic programming
sparse version, 38 engine
Polytope distance d (class), QPE basis inverse (class), 94
107 QPE full exact pricing (class),
portfolio optimization, 131–133 103
multi-period, 131, 132 QPE full filtered pricing
precision problem, 1 (class), 103
pricing, 18, 24–25, 86 QPE partial exact pricing
pricing strategy, 81, 82, 95–105 (class), 103
filtered pricing, 98–101 QPE partial filtered
implementation, 101–105 pricing (class), 103
partial filtered pricing, QPE pricing strategy (class),
103–105 101, 102
partial pricing, 97–98, 102 QPE solver (class), 82, 83, 85,
primal version 94, 101
polytope distance, 37, 110 QPErep (class), 83, 84, 86
smallest enclosing annulus, quadratic program, 15–16, 17,
42, 114, 126 35, 81, 86, 88, 104
primitive operation, 3, 8, 45, degenerate, 86
50–58, 59 nondegenerate, 19
program standard form, 16–17
convex, 46 unconstrained, 21
linear, 15, 17, 81, 88, 89, 94, quadratic programming, 3, 15,
98 35, 98, 105
quadratic, 15–16, 17, 35, 81, basis, 17, see QP-basis
86, 88, 104 solver, 3
programming quadratic programming engine,
convex, 8, 45 81–105
generic, 65, 66–69, 79, 81, basis inverse, 81, 82, 87–95
105 implementation, 94–95
154 Index

rational representation, second order surface, 49

88–89, 94, 96 simplex method, 16, 17–18, 35,
update, 89–94 85, 88, 89
design goals, 81–82 basis, 17, 18
number type, 82 basis matrix, 18
pricing strategy, 81, 82, generalized, 87
95–105 pivot step, 18
filtered pricing, 98–101 pricing, 18
implementation, 101–105 ratio test, 18
partial filtered pricing, reduced costs, 18
103–105
update, 18
partial pricing, 97–98, 102
size
solver, 81, 82–87, 94, 98,
of QP-basis, 22–23
101, 102, 103
slack variable, 16, 17, 85, 89, 90
access to original problem,
83–84 smallest enclosing annulus,
compile time tags, 84–86, 4, 42–43, 113–114,
94, 105 123–126, 135
exact arithmetic, 86 dual version, 42, 114, 126
symbolic perturbation, primal version, 42, 114, 126
86–87 smallest enclosing ball, 4, 6, 7,
39–41, 45, 83, 111–112,
119–123
— R — sparse version, 41, 121
ratio test, 18, 25–30, 86 smallest enclosing circle, 3
step 1, 25–27 smallest enclosing ellipse, 3, 50,
step 2, 27–28 58, 59, 126–127
step 3, 28–30 smallest enclosing ellipsoid, 7–8,
rational representation 45, 46
basis inverse, 88–89, 94, 96 SmEll (algorithm), 48, 49, 50,
realizing points, 108 52, 59
reduced costs, 18 solution
representation class, 72, 82, 83
basic, 21–23
revised simplex method, see
basic feasible, 17, 18, 21, 22
simplex method
feasible, 16, 21, 22
roundness test, 4
optimal, 16, 21, 22
solver, 35, 81, 82–87, 94, 98,
— S — 101, 102, 103
scenario, geometric optimization, access to original problem,
5–6 83–84
Index 155

compile time tags, 84–86, 94, basic, 17, 21, 22, 86, 89
105 entering, 18, 86, 89, 90, 95,
exact arithmetic, 86 97, 98, 99, 102, 103, 104
quadratic programming, 3 leaving, 18, 90, 103
standard, 5–6 nonbasic, 17, 18, 21, 22, 89,
symbolic perturbation, 86–87 95, 96, 97, 98, 102, 103,
sparse version 104
polytope distance, 38 original, 89, 90
smallest enclosing ball, 41, slack, 16, 17, 89, 90
121 variables-to-constraints ratio, 97
sparsity conditions, 5
standard form — W —
of quadratic program, 16–17
standard solver, 5–6 Welzl’s method, 3, 8, 9, 45,
Stl, 67, 68, 75, 76, 79, 81 47–50, 58, 59
stylus calibration, 133–137
StylusCalibration
(algorithm), 136
support points, 47, 50, 107, 111,
113
support set, 47, 58, 107, 108,
111, 112, 113
symbolic perturbation, 35, 86–87
symmetric objective matrix, 84,
85, 88, 110, 112

— T —
test environment, 117–118
traits class, 70, 76–79, 80, 82,
107, 111, 113
Triangulation 2 (class), 77

— U —
update, 18, 31
basis inverse, 89–94

— V —
variable
active, 97, 102, 103, 104
Curriculum Vitae

Sven Rolf Schönherr

born on March 17th , 1968, in Berlin, Germany
citizen of Germany

Education
1998 – 2002 Swiss Federal Institute of Technology Zurich /
Freie Universität Berlin, Germany
Ph.D. Studies in Computer Science
Candidate for Doctor of Technical Sciences

1987 – 1994 Freie Universität Berlin, Germany /

University of Technology Berlin, Germany
Studies of Mathematics and Computer Science
Diploma in Mathematics (FU Berlin, grade: 1.7)

1980 – 1987 Beethoven-Gymnasium (high school), Berlin, Germany

Allgemeine Hochschulreife (grade: 1.4)

Experience

since Mayfield Technologies, Waltersdorf (Berlin), Germany

04/2001 Software Developer / Project Leader
Conception, design, implementation, and certification
of a complex application for medical navigation systems

2000 – 2001 Swiss Federal Institute of Technology Zurich

Research Assistant / Teaching Assistant
in the group of Prof. Welzl
Design and implementation of a software library of
geometric data structures and algorithms within the
scope of a project funded by the European Community
and the Swiss Federal Office for Education and Science

1995 – 1999 Freie Universität Berlin, Germany

Scientific Programmer / Research Assistant
in the group of Prof. Welzl and Prof. Alt
 Design and implementation of a software library of
geometric data structures and algorithms within the
scope of several projects funded by the European
Community

1990 – 1992 Freie Universität Berlin, Germany

Student Assistant
in the group of Prof. Welzl
Implementation of geometric and optimization algo-
rithms

1989 – 1994 Self-employed Software Developer

Design and realization of several software projects