Loop Invariants: Analysis, Classification, and Examples

CARLO A. FURIA, ETH Zurich

BERTRAND MEYER, ETH Zurich, ITMO St. Petersburg, and Eiffel Software
SERGEY VELDER, ITMO St. Petersburg

Software verification has emerged as a key concern for ensuring the continued progress of information
technology. Full verification generally requires, as a crucial step, equipping each loop with a “loop invariant.”
Beyond their role in verification, loop invariants help program understanding by providing fundamental
insights into the nature of algorithms. In practice, finding sound and useful invariants remains a challenge.
Fortunately, many invariants seem intuitively to exhibit a common flavor. Understanding these fundamental
invariant patterns could therefore provide help for understanding and verifying a large variety of programs.
We performed a systematic identification, validation, and classification of loop invariants over a range
of fundamental algorithms from diverse areas of computer science. This article analyzes the patterns, as
uncovered in this study, governing how invariants are derived from postconditions; it proposes a taxonomy
of invariants according to these patterns; and it presents its application to the algorithms reviewed. The
discussion also shows the need for high-level specifications based on “domain theory.” It describes how the
invariants and the corresponding algorithms have been mechanically verified using an automated program
prover; the proof source files are available. The contributions also include suggestions for invariant inference
and for model-based specification.
Categories and Subject Descriptors: D.2.4 [Software/Program Verification]: Correctness Proofs; F.3.1
[Specifying and Verifying and Reasoning about Programs]: Invariants
General Terms: Algorithms, Verification
34
Additional Key Words and Phrases: Loop invariants, deductive verification, preconditions and postconditions,
formal verification
ACM Reference Format:
Carlo A. Furia, Bertrand Meyer, and Sergey Velder. 2014. Loop invariants: Analysis, classification, and
examples. ACM Comput. Surv. 46, 3, Article 34 (January 2014), 51 pages.
DOI: http://dx.doi.org/10.1145/2506375

1. INTRODUCTION: INDUCTIVE INVARIANTS

The problem of guaranteeing program correctness remains one of the central challenges
of software engineering, of considerable importance to the information technology in-
dustry and to society at large, which increasingly depends, for almost all of its pro-
cesses, on correctly functioning programs. As defined by Tony Hoare [2003], the “Grand

The ETH part of this work was partially supported by the Swiss National Science Foundation under projects
ASII (# 200021-134976) and LSAT (# 200020-134974), and by the Hasler Foundation under project Mancom.
The NRU ITMO part of this work was performed in the ITMO Software Engineering Laboratory with
financial support from the mail.ru group.
Authors’ addresses: C. Furia and B. Meyer, Software Engineering, Meyer, ETH Zentrum, Clausiusstrasse
59, 8092 Zurich, Switzerland and S. Velder, Software Engineering Laboratory, ITMO National Research
University, Kronversky Prospekt 49, Saint Petersburg 197101, Russia.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or commercial advantage and that
copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for
components of this work owned by others than ACM must be honored. Abstracting with credit is permitted.
To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this
work in other works requires prior specific permission and/or a fee. Permissions may be requested from
Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212)
869-0481, or [email protected].
c 2014 ACM 0360-0300/2014/01-ART34 $15.00
DOI: http://dx.doi.org/10.1145/2506375

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:2 C. A. Furia et al.

Challenge of Program Verification” mobilizes many researchers and practitioners using

a variety of techniques.
Some of these techniques, such as model checking [Clarke et al. 1999] and abstract
interpretation [Cousot and Cousot 1977], are directed at finding specific errors, such
as the possible violation of a safety property. An advantage of these techniques is that
they work on programs as they are, without imposing a significant extra annotation
effort on programmers. For full functional correctness—the task of proving that a pro-
gram satisfies a complete specification—the approach of choice remains, for imperative
programs, the Floyd-Hoare-Dijkstra style of axiomatic semantics. In this approach,
programs must be equipped with annotations in the form of assertions. Every loop, in
particular, must have a loop invariant.
Finding suitable loop invariants is a crucial and delicate step to verification. Although
some programmers may see invariant elicitation as a chore needed only for formal
verification, the concept is in fact widely useful, including for informal development:
the invariant gives fundamental information about a loop, showing what it is trying to
achieve and how it achieves it, to the point that (in some people’s view at least) it is
impossible to understand a loop without knowing its invariant.
To explore and illustrate this view, we have investigated a body of representative loop
algorithms in several areas of computer science to identify the corresponding invariants
and found that they follow a set of standard patterns. We set out to uncover, catalog,
classify, and verify these patterns and report our findings in the present article.
Finding an invariant for a loop is traditionally the responsibility of a human: either
the person performing the verification or the programmer writing the loop in the first
place (a better solution, when applicable, is the constructive approach to programming
advocated by Dijkstra and others [Dijkstra 1976; Gries 1981; Meyer 1980]). More
recently, techniques have been developed for inferring invariants automatically, or
semiautomatically with some human help (we review them in Section 5). We hope that
the results reported here will be useful in both cases: for humans, to help obtain the
loop invariants of new or existing programs, a task that many programmers still find
challenging; and for invariant inference tools.
For all algorithms presented in the article,1 we wrote fully annotated implementa-
tions and processed the result with the Boogie program verifier [Leino 2008], providing
proofs of correctness. The Boogie implementations are available at:2
http://bitbucket.org/sechairethz/verified/
This verification result reinforces the confidence in the correctness of the algorithms
presented in the article and their practical applicability.
The rest of this introductory section recalls the basic properties of invariants.
Section 2 introduces a style of expressing invariants based on “domain theory,” which
can often be useful for clarity and expressiveness. Section 3 presents two independent
classifications of loop invariant clauses, according to their role and syntactic similar-
ity with respect to the postcondition. Section 4 presents 21 algorithms from various
domains; for each algorithm, it presents an implementation in pseudo-code annotated
with complete specification and loop invariants. Section 5 discusses some related tech-
niques to infer invariants or other specification elements automatically. Section 6 draws
lessons from the verification effort. Section 7 concludes.

1 With the exception of those in Sections 4.5 and 4.7, whose presentation is at a higher level of abstraction,
so that a complete formalization would have required complex axiomatization of geometric and numerical
properties beyond the focus of this article.
2 In the repository, the branch inv_survey contains only the algorithms described in the article; see
http://goo.gl/DsdrV for instruction on how to access it.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:3

1.1. Loop Invariants Basics

The loop invariants of the axiomatic approach go back to Floyd [1967] and Hoare
[1969] (see Hatcliff et al. [2012] for a survey of notations for and variants of the
fundamental idea). For this approach and for the present article, a loop invariant is
not just a quantity that remains unchanged throughout executions of the loop body (a
notion that has also been studied in the literature), but more specifically an “inductive
invariant,” of which the precise definition appears next. Program verification also uses
other kinds of invariants, notably class invariants [Hoare 1972; Meyer 1997], which
the present discussion surveys only briefly in Section 1.4.
The notion of loop invariant is easy to express in the following loop syntax taken
from Eiffel:
1 from
2 Init
3 invariant
4 Inv
5 until
6 Exit
7 variant
8 Var
9 loop
10 Body
11 end

(the variant clause helps establish termination as discussed later). Init and Body are
each a compound (a list of instructions to be executed in sequence); either or both can be
empty, although Body normally will not. Exit and Inv (the inductive invariant) are both
Boolean expressions, that is to say, predicates on the program state. The semantics of
the loop is:
(1) Execute Init.
(2) Then, if Exit has value True, do nothing; if it has value False, execute Body, and
repeat step 2.
Another way of stating this informal specification is that the execution of the loop body
consists of the execution of Init followed by zero or more executions of Body, stopping
as soon as Exit becomes True.
There are many variations of the loop construct in imperative programming lan-
guages: “while” forms, which use a continuation condition rather than the inverse exit
condition; “do-until” forms, which always execute the loop body at least once, testing
for the condition at the end rather than on entry; and “for” or “do” forms (“across”
in Eiffel), which iterate over an integer interval or a data structure. They can all be
derived in a straightforward way from the previously mentioned basic form, on which
we will rely throughout this article.
The invariant Inv plays no direct role in the informal semantics but serves to reason
about the loop and its correctness. Inv is a correct invariant for the loop if it satisfies
the following conditions:
(1) Every execution of Init, started in the state preceding the loop execution, will yield
a state in which Inv holds.
(2) Every execution of Body, started in any state in which Inv holds and Exit does not
hold, will yield a state in which Inv holds again.
If these properties hold, then any terminating execution of the loop will yield a state
in which both Inv and Exit hold. This result is a consequence of the loop semantics,

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:4 C. A. Furia et al.

which defines the loop execution as the execution of Init followed by zero or more
executions of Body, each performed in a state where Exit does not hold. If Init ensures
satisfaction of the invariant, and any one execution of Body preserves it (it is enough
to obtain this property for executions started in a state not satisfying Exit), then Init
followed by any number of executions of Body will.
Formally, the following classic inference rule [Hoare 1972; Meyer 1997] uses the
invariant to express the correctness requirement on any loop:
{P} Init {Inv}, {Inv ∧ ¬Exit} Body {Inv}
.
{P} from Init until Exit loop Body end {Inv ∧ Exit}
This is a partial correctness rule, useful only for loops that terminate. Proofs of termi-
nation are in general handled separately through the introduction of a loop variant: a
value from a well-founded set, usually taken to be the set of natural numbers, which de-
creases upon each iteration (again, it is enough to show that it does so for initial states
not satisfying Exit). Since in a well-founded set all decreasing sequences are finite,
the existence of a variant expression implies termination. The rest of this discussion
concentrates on the invariants; it only considers terminating algorithms, of course, and
includes the corresponding variant clauses, but does not explain why the corresponding
expressions are indeed loop variants (nonnegative and decreasing). Invariants, how-
ever, also feature in termination proofs, where they ensure that the variant ranges over
a well-founded set (or, equivalently, the values it takes are bounded from below).
If a loop is equipped with an invariant, proving its partial correctness means estab-
lishing the two hypotheses in the aforementioned rules:
—{P} Init {Inv}, stating that the initialization ensures the invariant, is called the
initiation property.
—{Inv ∧ ¬Exit} Body {Inv}, stating that the body preserves the invariant, is called the
consecution (or inductiveness) property.
1.2. A Constructive View
We may look at the notion of loop invariant from the constructive perspective of a
programmer directing his or her program to reach a state satisfying a certain desired
property, the postcondition. In this view, program construction is a form of problem solv-
ing, and the various control structures are problem-solving techniques [Dijkstra 1976;
Meyer 1980; Gries 1981; Morgan 1994]; a loop solves a problem through successive
approximation.
The idea of this solution, illustrated in Figure 1, is the following:
—Generalize the postcondition (the characterization of possible solutions) into a
broader condition: the invariant.
—As a result, the postcondition can be defined as the combination (“and” in logic,
intersection in the figure) of the invariant and another condition: the exit condition.
—Find a way to reach the invariant from the previous state of the computation: the
initialization.
—Find a way, given a state that satisfies the invariant, to get to another state, still
satisfying the invariant but closer, in some appropriate sense, to the exit condition:
the body.
For the solution to reach its goal after a finite number of steps, we need a notion of
discrete “distance” to the exit condition. This is the loop variant.
The importance of this presentation of the loop process is that it highlights the nature
of the invariant: it is a generalized form of the desired postcondition, which in a special
case (represented by the exit condition) will give us that postcondition. This view of the

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:5

Fig. 1. The loop as a computation by approximation.

invariant, as a particular way of generalizing the desired goal of the loop computation,
explains why the loop invariant is such an important property of loops; one can argue
that understanding a loop means understanding its invariant (in spite of the obvious
observation that many programmers write loops without ever formally learning the
notion of invariant, although we may claim that if they understand what they are
doing they are relying on some intuitive understanding of the invariant anyway, like
Molière’s Mr. Jourdain speaking in prose without knowing it).
The key results of this article can be described as generalization strategies to obtain
invariants from postconditions.

1.3. A Basic Example

To illustrate these ideas, the 2,300-year-old example of Euclid’s algorithm, while very
simple, is still a model of elegance. The postcondition of the algorithm is
Result = gcd(a, b),
where the positive integers a and b are the input and gcd is the mathematical Greatest
Common Divisor function. The generalization is to replace this condition by
Result = x ∧ gcd(Result, x) = gcd(a, b) (1)
with a new variable x, taking advantage of the mathematical property that, for every x,

gcd(x, x) = x. (2)
The second conjunct, a generalization of the postcondition, will serve as the invariant;
the first conjunct will serve as the exit condition. To obtain the loop body, we take
advantage of another mathematical property: for every x > y,
gcd(x, y) = gcd(x − y, y), (3)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:6 C. A. Furia et al.

Fig. 2. Greatest common divisor with substraction.

yielding the well-known algorithm in Figure 2. (As with any assertion, writing clauses
successively in the invariant is equivalent to a logical conjunction.) This form of Euclid’s
algorithm uses subtraction; another form, given in Section 4.2.2, uses integer division.
We may use this example to illustrate some of the orthogonal categories in the
classification developed in the rest of this article:
—The last clause of the invariant is an essential invariant, representing a weakening of
the postcondition. The first two clauses are a bounding invariant, indicating that the
state remains within certain general boundaries and ensuring that the “essential”
part is defined.
—The essential invariant is a conservation invariant, indicating that a certain quantity
remains equal to its original value.
—The strategy that leads to this conservation invariant is uncoupling, which replaces
a property of one variable (Result), used in the postcondition, by a property of two
variables (Result and x), used in the invariant.
The proof of correctness follows directly from the mathematical property stated:
(2) establishes initiation, and (3) establishes consecution.
Section 4.2.2 shows how the same technique is applicable backward to guess likely
loop invariants given an algorithm annotated with pre- and postcondition: mutating
the latter yields a suitable loop invariant.
1.4. Other Kinds of Invariants
Loop invariants are the focus of this article, but before we return to them it is useful
to list some other kinds of invariants encountered in software. (Yet other invariants,
which lie even further beyond the scope of this discussion, play fundamental roles in
fields such as physics; consider, for example, the invariance of the speed of light under
a Lorentz transformation and of time under a Galilean transformation.)
In object-oriented programming, a class invariant (also directly supported by the
Eiffel notation [Eiffel 2006]) expresses a property of a class that:
—Every instance of the class possesses immediately after creation, and
—Every exported feature (operation) of the class preserves,
with the consequence that whenever such an object is accessible to the rest of the
software it satisfies the invariant, since the life of an object consists of creation followed
by any number of “qualified” calls x . f to exported features f by clients of the class.
ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:7

The two properties listed are strikingly similar to initiation and consecution for loop
invariants, and the connection appears clearly if we model the life of an object as a loop:
1 from
2 create x.make - - Written in some languages as x := new C()
3 invariant
4 CI - - The class invariant
5 until
6 “x is no longer needed”
7 loop
8 x.some feature of the class
9 end
Also useful are Lamport-style invariants [Lamport 1977] used to reason about con-
current programs, which obviate the need for the “ghost variables” of the Owicki-Gries
method [Owicki and Gries 1976]). Like other invariants, a Lamport invariant is a pred-
icate on the program state; the difference is that the definition of the states involves not
only the values of the program’s variables but also the current point of the execution of
the program (“Program Counter” or PC) and, in the case of a concurrent program, the
collection of the PCs of all its concurrent processes. An example of application is the
answer to the following problem posed by Lamport [2009].
Consider N processes numbered from 0 through N − 1 in which each process i executes
i0 : x[i] := 1
i1 : y[i] := x[(i − 1) mod N]
i2 :
and stops where each x[i] initially equals 0. (The reads and writes of each x[i] are assumed to be atomic.)
[. . .] The algorithm [. . .] maintains an inductive invariant. Do you know what that invariant is?

If we associate a proposition @(m, i) for m = 1, 2, 3 that holds precisely when the execu-
tion of process i reaches location im, an invariant for the algorithm can be expressed as:
⎛ ⎞
@(0, (i − 1) mod N) ∧ y[i] = 0
⎜ ∨ ⎟
⎜ ⎟
@(2, i) =⇒ ⎜ @(1, (i − 1) mod N) ∧ y[i] = 1 ⎟ .
⎝ ∨ ⎠
@(2, (i − 1) mod N) ∧ y[i] = 1
Yet another kind of invariant occurs in the study of dynamical systems, where an
invariant is a region I ⊆ S of the state space S such that any trajectory starting in I or
entering it stays in I indefinitely in the future:
∀x ∈ I, ∀t ∈ T : (t, x) ∈ I,
where T is the time domain and  : T × S → S is the evolution function. The connection
between dynamical system invariants and loop invariants is clear in the constructive
view (Section 1.2) and can be formally derived by modeling programs as dynamical
systems or using some other operational formalism [Furia et al. 2012]. The differential
invariants introduced in the study of hybrid systems [Platzer 2010] are also variations
of the invariants defined by dynamical systems.
2. EXPRESSING INVARIANTS: DOMAIN THEORY
To discuss and compare invariants, we need to settle on the expressiveness of the
underlying invariant language: what do we accept as a loop invariant?
The question involves general assertions, not just invariants; more generally, we
must make sure that any convention for invariants is compatible with the general
ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:8 C. A. Furia et al.

scheme used for pre-/postspecification, since an invariant is a mutation (possibly a

weakening) of the postcondition.
The mathematical answer to the basic question is simple: an assertion other than a
routine postcondition, in particular a loop invariant, is a predicate on the program state.
For example, the assertion x > 0, where x is a program variable, is the predicate that
holds of all computation states in which the value of that variable is positive. (Another
view would consider it as the subset of the state space containing all states that satisfy
the condition; the two views are equivalent since the predicate is the characteristic
function of the subset, and the subset is the inverse domain of “true” for the predicate.)
A routine postcondition is usually a predicate on two states, since the specification
of a routine generally relates new values to original ones. For example, an increment
routine yields a state in which the counter’s value is one more on exit than on entry. The
old notation, available for postconditions in Eiffel and other programming languages
supporting contracts, reflects this need; for example, a postcondition clause could read
counter = old counter + 1. Other notations, notably the Z specification language [Spivey
1992], have a notation for “new” rather than “old,” as in counter’ = counter + 1 where
the primed variable denotes the new value. Although invariants are directly related
to postconditions, we will be able in this discussion to avoid such notations and treat
invariants as one-state functions. (Technically, this is always possible by recording the
entry value as part of the state.)
Programming languages offer a mechanism directly representing predicates on
states: Boolean expressions. This construct can therefore be used—as in the x > 0
example—to represent assertions; this is what assertion-aware programming lan-
guages typically do, often extending it with special notations such as old and support
for quantifiers.
This basic language decision leaves open the question of the level of expressiveness
of assertions. There are two possibilities:

—Allow assertions, in particular postconditions and loop invariants, to use functions

and predicates defined using some appropriate mechanism (often, the programming
language’s function declaration construct) to express high-level properties based on
a domain theory covering specifics of the application area. We call this approach
domain theory.3
—Disallow this possibility, requiring assertions always to be expressed in terms of the
constructs of the assertion language, without functions. We call this approach atomic
assertions.

The example of Euclid’s algorithm given earlier, simple as it is, was already an
example of the domain-theory-based approach because of its use of a function gcd in
the invariant clause
gcd(Result, x) = gcd(a, b) (4)
corresponding to a weakening of the routine postcondition

Result = gcd(a, b).

It is possible to do without such a function by going back to the basic definition of

the greatest common denominator. In such an atomic-assertion style, the postcondition
would read

3 No relation with the study of partially ordered sets, also called domain theory [Abramsky and Jung 1994].

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:9

Result > 0 (Alternatively, Result ≥ 1)

a \\ Result = 0 (Result divides a)
b \\ Result = 0 (Result divides b)
∀i ∈ N : (a \\ i = 0) ∧ (b \\ i = 0) implies i ≤ Result (Result is the greatest of all the numbers
that satisfy the preceding properties).

Expressing the invariant in the same style requires several more lines since the defi-
nition of the greatest common divisor must be expanded for both sides of (4).
Even for such a simple example, the limitations of the atomic-assertion style are
clear: because it requires going back to basic logical constructs every time, it does not
scale.
Another example where we can contrast the two styles is any program that computes
the maximum of an array. In the atomic-assertion style, the postcondition will be
written

∀k ∈ Z : a.lower ≤ k ≤ a.upper implies a[k] ≤ Result (Every element between bounds has a
value smaller than Result)
∃k ∈ Z : a.lower ≤ k ≤ a.upper ∧ a[k] = Result (Some element between bounds has the
value Result).

This property is the definition of the maximum and hence needs to be written some-
where. If we define a function “max” to capture this definition, the specification becomes
simply
Result = max(a).
The difference between the two styles becomes critical when we come to the invari-
ant of programs computing an array’s maximum. Two different algorithms appear in
Section 4.1. The first (Section 4.1.1) is the most straightforward; it moves an index i
from a.lower + 1 to a.upper, updating Result if the current value is higher than the
current result (initialized to the first element a [a.lower]). With a domain theory on
arrays, the function max will be available as well as a notion of slice, where the slice
a [i..j] for integers i and j is the array consisting of elements of a in the range [i, j].
Then the invariant is simply
Result = max(a [a.lower..i]),
which is ensured by initialization and, on exit when i = a.upper, yields the postcondition
Result = max(a) (based on the domain theory property that a [a.lower..a.upper] = a).
The atomic-assertion invariant would be a variation of the expanded postcondition:
∀k ∈ Z : a.lower ≤ k ≤ i implies a[k] ≤ Result
∃k ∈ Z : a.lower ≤ k ≤ i ∧ a[k] = Result.
Consider now a different algorithm for the same problem (Section 4.1.2), which works
by exploring the array from both ends, moving the left cursor i up if the element at
i is less than the element at j and otherwise moving the right cursor j down. The
atomic-assertion invariant can be written with an additional level of quantification:

∀k ∈ Z : a.lower ≤ k ≤ a.upper implies a[k] ≤ m
∃m : . (5)
∃k ∈ Z : i ≤k≤ j and a[k] = m
Alternatively, we can avoid quantifier alternation using the characterization based on
the complement property that the maximal element is not outside the slice a[i..j]:
∀k ∈ Z : a.lower ≤ k < i ∨ j < k ≤ a.upper =⇒ a[k] ≤ a[i] ∨ a[k] ≤ a[ j]. (6)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:10 C. A. Furia et al.

The form without quantifier alternation is more amenable to automated reasoning,

but it has the disadvantage that it requires additional ingenuity and is not a straight-
forward modification of the invariant for the one-way version of the algorithm. More
significantly for this article’s point of view, both formulations (5) and (6) give an appear-
ance of complexity even though the invariant is conceptually very simple, capturing in
a nutshell the essence of the algorithm (as noted earlier, one of the applications of a
good invariant is that it enables us to understand the core idea behind a loop):
max(a) = max(a[i.. j]). (7)
In words: the maximum of the entire array is to be found in the slice that has not been
explored yet. On exit, where i = j, we are left with a one-element slice, whose value
(this is a small theorem of the corresponding domain theory) is its maximum and hence
the maximum of the whole array. The domain theory invariant makes the algorithm
and its correctness immediately clear.
The domain theory approach means that, before any attempt to reason about an al-
gorithm, we should develop an appropriate model of the underlying domain by defining
appropriate concepts such as greatest common divisor for algorithms on integers and
slices and maximum for algorithms on arrays, establishing the relevant theorems (e.g.,
that x > y =⇒ gcd(x, y) = gcd(x − y, y) and that max(a[i..i]) = a[i]). These concepts
and theorems need only be developed once for every application domain of interest, not
anew for every program over that domain. The programs can then use the correspond-
ing functions in their assertions, in particular in the loop invariants.
The domain theory approach takes advantage of the standard abstraction mecha-
nism of mathematics. Its only practical disadvantage, for assertions embedded in a
programming language, is that the functions over a domain (such as gcd) must come
from some library and, if themselves written in the programming language, must sat-
isfy strict limitations; in particular, they must be “pure” functions defined without any
reference to imperative constructs. This issue only matters, however, for the practical
embedding of invariants in programs; it is not relevant to the conceptual discussion
of invariants, independent of any implementation concerns, which is the focus of this
article.
For the same reason, this article does not explore—except for Section 6—the often
delicate tradeoff between succinctness of expression and amenability to automated
reasoning. For example, the invariant (5) is concisely captured as (7) in domain theory
form even if it uses quantifier alternation; the different formulation (6) is not readily
expressible in terms of slice and maximum functions, but it may be easier to han-
dle by automatic theorem provers since complexity grows with quantifier alternation
[Papadimitriou 1993]. This article’s focus is on developing and understanding the
essence of algorithms through loop invariants presented at the right level of abstrac-
tion, largely independent of the requirements posed by automated reasoning. Section 6,
however, demonstrates that the domain theory approach is still practically applicable.
The remainder of this article relies, for each class of algorithms, on the appropriate
domain theory, whose components (functions and theorems) are summarized at the
beginning of the corresponding section. We will make no further attempt at going back
to the atomic-assertion style; the previous examples should suffice to show how much
simplicity is gained through this policy.

3. CLASSIFYING INVARIANTS
Loop invariants and their constituent clauses can be classified along two dimensions:
—By their role with respect to the postcondition (Section 3.1), leading us to distinguish
between “essential” and “bounding” invariant properties.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:11

—By the transformation technique that yields the invariant from the postcondition
(Section 3.2). Here we have techniques such as uncoupling and constant relaxation.
3.1. Classification by Role
In the typical loop strategy described in Section 1.2, it is essential that successive iter-
ations of the loop body remain in the convergence regions where the generalized form
of the postcondition is defined. The corresponding conditions make up the bounding
invariant; the clause describing the generalized postcondition is the essential invari-
ant. The bounding invariant for the greatest common divisor algorithm consists of the
clauses
Result > 0
x > 0.
The essential clause is
gcd(Result, x) = gcd(a, b),
yielding the postcondition when Result = x.
For the one-way maximum program, the bounding invariant is
a.lower ≤ i ≤ a.upper
and the essential invariant is
Result = max (a[a.lower..i]),
yielding the postcondition when i = a.upper. Note that the essential invariant would
not be defined without the bounding invariant, since the slice a[1..i] would be undefined
(if i > a.upper) or would be empty and have no maximum (if i < a.lower).
For the two-way maximum program, the bounding invariant is
a.lower ≤ i ≤ j ≤ a.upper
and the essential invariant is
max(a) = max(a[i.. j]),
yielding the postcondition when i = j. Again, the essential invariant would not be
always defined without the bounding invariant.
The separation between bounding and essential invariants is often straightforward
as in these examples. In case of doubt, the following observation will help distinguish.
The functions involved in the invariant (and often, those of the postcondition) are often
partial; for example:
—gcd(u, v) is only defined if u and v are both nonzero (and, since we consider natural
integers only in the example, positive).
—For an array a and an integer i, a[i] is only defined if i ∈ [a.lower..a.upper], and the
slice a[i..j] is nonempty only if [i..j] ⊆ [a.lower..a.upper].
—max(a) is only defined if the array a is not empty.
Since the essential clauses, obtained by postcondition generalization, use gcd (Result,
x) and (in the array algorithms) array elements and maxima, the invariants must
include the bounding clauses as well to ensure that these essential clauses are
meaningful. A similar pattern applies to most of the invariants studied later.
3.2. Classification by Generalization Technique
The essential invariant is a mutation (often, a weakening) of the loop’s postcondition.
The following mutation techniques are particularly common:

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:12 C. A. Furia et al.

Constant relaxation: replace a constant n (more generally, an expression which does

not change during the execution of the algorithm) by a variable i, and use i = n
as part or all of the exit condition.
Constant relaxation is the technique used in the one-way array maximum computa-
tion, where the constant is the upper bound of the array. The invariant generalizes the
postcondition “Result is the maximum of the array up to a.lower,” where a.lower is a
constant, with “Result is the maximum up to i.” This condition is trivial to establish
initially for a nonempty array (take i to be a.lower), is easy to extend to an incremented
i (take Result to be the greater of its previous value and a[i]), and yields the postcondi-
tion when i reaches a.upper. As we will see in Section 4.1.4, binary search differs from
sequential search by applying double constant relaxation to both the lower and upper
bounds of the array.
Uncoupling: replace a variable v (often Result) by two (e.g., Result and x), using
their equality as part or all of the exit condition.
Uncoupling is used in the greatest common divisor algorithm.
Term dropping: remove a subformula (typically a conjunct), which gives a straight-
forward weakening.
Term dropping is used in the partitioning algorithm (Section 4.3.1).
Aging: replace a variable (more generally, an expression) by an expression that
represents the value the variable had at previous iterations of the loop.
Aging typically accommodates “off-by-one” discrepancies between when a variable is
evaluated in the invariant and when it is updated in the loop body.
Backward reasoning: compute the loop’s postcondition from another assertion by
backward substitution.
Backward reasoning can be useful for nested loops, where the inner loop’s postcondition
can be derived from the outer loop’s invariant.
4. THE INVARIANTS OF IMPORTANT ALGORITHMS
The following subsections include a presentation of several algorithms, their loop in-
variants, and their connection with each algorithm’s postcondition. Table I lists the
algorithms and their category. For more details about variants of the algorithms and
their implementation, we refer to standard textbooks on algorithms [Mehlhorn and
Sanders 2008; Cormen et al. 2009; Knuth 2011].
4.1. Array Searching
Many algorithmic problems can be phrased as search over data structures—from the
simple arrays up to graphs and other sophisticated representations. This section illus-
trates some of the basic algorithms operating on arrays.
4.1.1. Maximum: One-Variable Loop. The following routine max_one_way returns the
maximum element of an unsorted array a of bounds a.lower and a.upper. The max-
imum is only defined for a nonempty array; thus, the precondition a.count ≥ 1. The
postcondition can be written
Result = max(a).
Writing it in slice form, as Result = max (a [a.lower..a.upper]), yields the invariant
by constant relaxation of either of the bounds. We choose the second one, a.upper,

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:13

Table I. The Algorithms Presented in Section 4

ALGORITHM TYPE SECTION
Maximum search (one variable) searching §4.1.1
Maximum search (two variable) searching §4.1.2
Sequential search in unsorted array searching §4.1.3
Binary search searching §4.1.4
Integer division arithmetic §4.2.1
Greatest common divisor (with division) arithmetic §4.2.2
Exponentiation (by squaring) arithmetic §4.2.3
Long integer addition arithmetic §4.2.4
Quick sort’s partitioning sorting §4.3.1
Selection sort sorting §4.3.2
Insertion sort sorting §4.3.3
Bubble sort (basic) sorting §4.3.4
Bubble sort (improved) sorting §4.3.5
Comb sort sorting §4.3.6
Knapsack with integer weights dynamic programming §4.4.1
Levenstein distance dynamic programming §4.4.2
Rotating calipers algorithm computational geometry §4.5
List reversal data structures §4.6.1
Binary search trees data structures §4.6.2
PageRank algorithm fixpoint §4.7

Fig. 3. Maximum: one-variable loop.

yielding the essential invariant clause

Result = max(a [a.lower..i]).
Figure 3 shows the resulting implementation of the algorithm.
Proving initiation is trivial. Consecution relies on the domain theory property that
max(a [1..i + 1]) = max(max(a [1..i]), a [i + 1]).

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:14 C. A. Furia et al.

Fig. 4. Maximum: two-variable loop.

4.1.2. Maximum: Two-Variable Loop. The one-way maximum algorithm results from ar-
bitrarily choosing to apply constant relaxation to either a.lower or (as in the previous
version) a.upper. Guided by a symmetry concern, we may choose double constant relax-
ation, yielding another maximum algorithm max_two_way, which traverses the array
from both ends. If i and j are the two relaxing variables, the loop body either increases
i or decreases j. When i = j, the loop has processed all of a, and hence i and j indicate
the maximum element.
The specification (precondition and postcondition) is the same as for the previous
algorithm. Figure 4 shows an implementation.
It is again trivial to prove initiation. Consecution relies on the following two domain
theory properties:
j > i ∧ a [i] ≥ a [ j] =⇒ max(a [i.. j]) = max(a [i.. j − 1]) (8)
i < j ∧ a [ j] ≥ a [i] =⇒ max(a [i.. j]) = max(a [i + 1.. j]). (9)
4.1.3. Search in an Unsorted Array. The following routine has_sequential returns the
position of an occurrence of an element key in an array a or, if key does not appear, a
special value. The algorithm applies to any sequential structure but is shown here for
arrays. For simplicity, we assume that the lower bound a.lower of the array is 1, so that
we can choose 0 as the special value. Obviously this assumption is easy to remove for
generality: just replace 0, as a possible value for Result, by a.lower −1.
The specification may use the domain theory notation elements (a) to express the set
of elements of an array a. A simple form of the postcondition is
Result = 0 ⇐⇒ key ∈ elements(a), (10)
which just records whether the key has been found. We will instead use a form that
also records where the element appears if present:
Result = 0 =⇒ key = a[Result] (11)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:15

Fig. 5. Search in an unsorted array.

Result = 0 =⇒ key ∈ elements(a), (12)

to which we can for clarity prepend the bounding clause
Result ∈ [0..a.upper]
to make it explicit that the array access in (11) is defined when needed.
If in (12) we replace a by a [1..a.upper], we obtain the loop invariant of sequential
search by constant relaxation: introducing a variable i to replace either of the bounds
1 and a.upper. Choosing the latter yields the following essential invariant:
Result ∈ [0, i]
Result = 0 =⇒ key = a [Result]
Result = 0 =⇒ key ∈ elements(a [1..i]),
leading to an algorithm that works on slices [1..i] for increasing i, starting at 0 and
with bounding invariant 0 ≤ i ≤ a.count, as shown in Figure 5.4
To avoid useless iterations, the exit condition may be replaced by i = a.upper ∨
Result > 0.
To prove initiation, we note that initially Result is 0 and the slice a [1..i] is empty.
Consecution follows from the domain theory property that, for all 1 ≤ i ≤ a.upper:
key ∈ elements(a [1..i + 1]) ⇐⇒ key ∈ elements(a [1..i]) ∨ key = a[i + 1].

4 Notethat in this example it is OK for the array to be empty, so there is no precondition on a.upper, although
general properties of arrays imply that a.upper ≥ 0; the value 0 corresponds to an empty array.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:16 C. A. Furia et al.

4.1.4. Binary Search. Binary search works on sorted arrays by iteratively halving a
segment of the array where the searched element may occur. The search terminates
either when the element is found or when the segment becomes empty, implying that
the element appears nowhere in the array.
As already remarked by Knuth many years ago [Knuth 2011, Vol. 3, Sec. 6.2.1]:
Although the basic idea of binary search is comparatively straightforward, the details can be surprisingly
tricky, and many programmers have done it wrong the first few times they tried.

Reasoning carefully on the specification (at the domain theory level) and the resulting
invariant helps avoid mistakes.
For the present discussion, it is interesting that the postcondition is the same as for
sequential search (Section 4.1.3), so that we can see where the generalization strategy
differs, taking advantage of the extra property that the array is sorted.
The algorithm and implementation now have the precondition
sorted (a),
where the domain theory predicate sorted (a), defined as
∀ j ∈ [a.lower..a.upper − 1] : a [ j] ≤ a [ j + 1],
expresses that an array is sorted upward. The domain theorem on which binary search
rests is that, for any value mid in [i..j] (where i and j are valid indexes of the array),
and any value key of type T (the type of the array elements):
key ≤ a[mid] ∧ key ∈ elements(a[i..mid])
key ∈ elements(a[i.. j]) ⇐⇒ ∨ . (13)
key > a[mid] ∧ key ∈ elements(a[mid + 1.. j])
This property leads to the key insight behind binary search, whose invariant follows
from the postcondition by variable introduction, mid serving as that variable.
Formula (13) is not symmetric with respect to i and j; a symmetric version is possible,
using in the second disjunct, “≥” rather than “>” and mid rather than mid + 1. The
form given in (13) has the advantage of using two mutually exclusive conditions in the
comparison of key to a [mid]. As a consequence, we can limit ourselves to a value mid
chosen in [i..j − 1] (rather than [i..j]) since the first disjunct does not involve j and the
second disjunct cannot hold for mid = j (the slice a [mid + 1..j] being then empty). All
these observations and choices have direct consequences on the program text but are
better handled at the specification (theory) level.
We will start for simplicity with the version (10) of the postcondition that only records
presence or absence, repeated here for ease of reference:
Result = 0 ⇐⇒ key ∈ elements(a). (14)
Duplicating the right-hand side of (14), writing a in slice form a[1..a.upper], and ap-
plying constant relaxation twice, to the lower bound 1 and the upper bound a.upper,
yields the essential invariant:
key ∈ elements(a[i.. j]) ⇐⇒ key ∈ elements(a) (15)
with the bounding invariant
1 ≤ i ≤ mid + 1 ∧ 1 ≤ mid ≤ j ≤ a.upper ∧ i ≤ j,
which combines the assumptions on mid necessary to apply (13)—also assumed in
(15)—and the additional knowledge that 1 ≤ i and j ≤ a.upper.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:17

Fig. 6. Binary search.

The attraction of this presentation is that:

—The two clauses key ≤ a[mid] and key > a[mid] of (13) are easy-to-test complementary
conditions, suggesting a loop body that preserves the invariant by testing key against
a [mid] and going left or right as a result of the test.
—When i = j—the case that serves as exit condition—the left side of the equivalence
(15) reduces to key = a [i]; evaluating this expression tells us whether key appeared
at all in the entire array, the information we seek. In addition, we can obtain the
stronger postcondition, (11)–(12), which gives Result its precise value, by simply
assigning i to Result.
This leads to the implementation in Figure 6.
To prove initiation, we note that initially mid is 1, so that mid ∈ [i..j] is true.
Consecution follows directly from (13).
For the expression assigned to mid in the loop, given in pseudo-code as “A value in
[i..j − 1]”, the implementation indeed chooses, for efficiency, the midpoint of the interval
[i..j], which may be written i + (j − i) // 2, where “//” denotes integer division. In an im-
plementation, this form is to be preferred to the simpler (i + j) // 2, whose evaluation on
a computer may produce an integer overflow even when i, j, and their midpoint are all
correctly representable on the computer’s number system, but (because they are large)
the sum i + j is not [Bloch 2006]. In such a case the evaluation of j − i is instead safe.
4.2. Arithmetic Algorithms
Efficient implementations of the elementary arithmetic operations known since grade
school require nontrivial algorithmic skills and feature interesting invariants, as the
examples in this section demonstrate.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:18 C. A. Furia et al.

Fig. 7. Integer division.

4.2.1. Integer Division. The algorithm for integer division by successive differences com-
putes the integer quotient q and the remainder r of two integers m and n. The postcon-
dition reads
0≤r<m
n = m · q + r.

The loop invariant consists of a bounding clause and an essential clause. The latter
is simply an element of the postcondition:
n = m · q + r.
The bounding clause weakens the other postcondition clause by keeping only its first
part:
0 ≤ r,
so that the dropped condition r < m becomes the exit condition. As a consequence,
r ≥ m holds in the loop body, and the assignment r:= r − m maintains the invariant
property 0 ≤ r. It is straightforward to prove the implementation in Figure 7 correct
with respect to this specification.
4.2.2. Greatest Common Divisor (with Division). Euclid’s algorithm for the greatest com-
mon divisor offers another example where clearly separating between the underlying
mathematical theory and the implementation yields a concise and convincing correct-
ness argument. Sections 1.3 and 2 previewed this example by using the form that
repeatedly subtracts one of the values from the other; here we will use the version that
uses division.
The greatest common divisor gcd(a, b) is the greatest integer that divides both a and
b, defined by the following axioms, where a and b are nonnegative integers such that
at least one of them is positive (“\\” denotes integer remainder):
a\\ gcd(a, b) = 0
b\\ gcd(a, b) = 0
∀d ∈ N : (a\\d = 0) ∧ (b\\d = 0) =⇒ d ≤ gcd(a, b).

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:19

Fig. 8. Greatest common divisor with division.

From this definition follow several properties of the gcd function:

Commutativity: gcd(a, b) = gcd(b, a)
Zero divisor: gcd(a, 0) = a
Reduction: for b > 0, gcd(a, b) = gcd(a\\b, b)
The following property of the remainder operation is also useful:
Nonnegativity: for integers a ≥ 0 and b > 0: a\\b ≥ 0
From the obvious postcondition Result = gcd (a, b), we obtain the essential invariant
in three steps:
(1) By backward reasoning, derive the loop’s postcondition x = gcd(a, b) from the rou-
tine’s postcondition Result = gcd (a,b).
(2) Using the zero divisor property, rewrite it as gcd(x, 0) = gcd(a, b).
(3) Apply constant relaxation, introducing variable y to replace 0.
This gives the essential invariant gcd(x, y) = gcd(a, b) together with the bounding
invariants x > 0 and y ≥ 0. The corresponding implementation is shown in Figure 8.5
Initiation is established trivially. Consecution follows from the reduction property.
Note that, unlike in the difference version (Section 1.3), we can arbitrarily decide
always to divide x by y, rather than having to find out which of the two numbers is
greater; hence, the commutativity of gcd is not used in this proof.
4.2.3. Exponentiation by Successive Squaring. Suppose we do not have a built-in power
operator and wish to compute mn. We may of course multiply m by itself n − 1 times,

5 Thevariant is simply y, which is guaranteed to decrease at every iteration and can be bounded from below
by the property 0 ≤ x\\y < y.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:20 C. A. Furia et al.

Fig. 9. Exponentiation by successive squaring.

but a more efficient algorithm squares m for all 1s values in the binary representation
of n. In practice, there is no need to compute this binary representation.
Given the postcondition
Result = mn,
we first rewrite it into the obviously equivalent form Result · 11 = mn. Then, the
invariant is obtained by double constant relaxation: the essential property
Result · x y = mn
is easy to obtain initially (by setting Result, x, and y to 1, m, and n), yields the post-
condition when y = 0, and can be maintained while progressing toward this situation
thanks to the domain theory properties
x 2z = (x 2 )2z/2 (16)
x z = x · x z−1 . (17)
Using only (17) would lead to the inefficient (n − 1)-multiplication algorithm, but we
may use (16) for even values of y = 2z. This leads to the algorithm in Figure 9.
Proving initiation is trivial. Consecution is a direct application of the (16) and (17)
properties.
4.2.4. Long Integer Addition. The algorithm for long integer addition computes the sum
of two integers a and b given in any base as arrays of positional digits starting from
the least significant position. For example, the array sequence 3, 2, 0, 1 represents
the number 138 in base 5 as 3 · 50 + 2 · 51 + 0 · 52 + 1 · 53 = 138. For simplicity of
representation, in this algorithm we use arrays indexed by 0, so that we can readily

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:21

express the value encoded in base b by an array a as the sum:

a.count
a[k] · bk.
k=0

The postcondition of the long integer addition algorithm has two clauses. One speci-
fies that the pairwise sum of elements in a and b encodes the same number as Result:
n−1 n
(a[k] + b[k]) · basek = Result[k] · basek. (18)
k=0 k=0

Result may have one more digit than a or b, hence the different bound in the two sums,
where n denotes a’s and b’s length (normally written a.count and b.count). The second
postcondition clause is the consistency constraint that Result is indeed a representa-
tion in base base:
has base(Result, base), (19)
where the predicate has_base is defined by a quantification over the array’s length:
has base(v, b) ⇐⇒ ∀k ∈ N : 0 ≤ k < v.count =⇒ 0 ≤ v[k] < b.
Both postcondition clauses appear mutated in the loop invariant. First, we rewrite
Result in slice form Result [0..n] in (18) and (19). The first essential invariant clause
follows by applying constant relaxation to (19), with the variable expression i − 1
replacing constant n:
has base(Result[0..i − 1], base).
The decrement is required because the loop updates i at the end of each iteration; it is
a form of aging (see Section 3.2).
To get the other part of the essential invariant, we first highlight the last term in
the summation on the right-hand side of (18):
n−1 n−1
(a[k] + b[k]) · basek = Result[n] · basen + Result[k] · basek.
k=0 k=0

We then introduce variables i and carry, replacing constants n and Result[n]. Variable
i is the loop counter, also mentioned in the other invariant clause; carry, as the name
indicates, stores the remainder of each pairwise addition, which will be carried over to
the next digit.
The domain property that the integer division by b of the sum of two b-base digits
v1 , v2 is less than b (all variables are integer):
b > 0 ∧ v1 , v2 ∈ [0..b − 1] =⇒ (v1 + v2 )//b ∈ [0..b − 1]
suggests the bounding invariant clause 0 ≤ carry < base. Figure 10 shows the re-
sulting implementation, where the most significant digit is set after the loop before
terminating.
Initiation is trivial under the convention that an empty sum evaluates to zero. Con-
secution easily follows from the domain theoretic properties of the operations in the
loop body, and in particular from how the carry and the current digit d are set in each
iteration.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:22 C. A. Furia et al.

Fig. 10. Long integer addition.

4.3. Sorting
A number of important algorithms sort an array based on pairwise comparisons and
swaps of elements. The following domain theory notations will be useful for arrays a
and b:
—perm (a,b) expresses that the arrays are permutations of each other (their elements
are the same, each occurring the same number of times as in the other array).
—sorted (a) expresses that the array elements appear in increasing order: ∀i ∈
[a.lower..a.upper − 1] : a[i] ≤ a[i + 1].
The sorting algorithms considered sort an array in place, with the specification:
sort(a : ARRAY [T ])
require
a.lower = 1
a.count = n ≥ 1
ensure
perm (a, old a)
sorted (a)
The type T indicates a generic type that is totally ordered and provides the comparison
operators <, ≤, ≥, and >. The precondition that the array be indexed from 1 and

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:23

nonempty is a simplification that can be easily dropped; we adopt it in this section as

it focuses the presentation of the algorithms on the interesting cases. For brevity, we
also use n as an alias of a’s length a.count.
The notation a[i.. j] ∼ x, for an array slice a [i..j], a scalar value x, and a comparison
operator ∼ among <, ≤, ≥, and >, denotes that all elements in the slice satisfy ∼ with
respect to x: it is a shorthand for ∀k ∈ [i.. j]: a[k] ∼ x.
4.3.1. Quick Sort: Partitioning. At the core of the well-known quick sort algorithm lies the
partitioning procedure, which includes loops with an interesting invariant; we analyze
it in this section.
The procedure rearranges the elements in an array a according to an arbitrary value
pivot given as input: all elements in positions up to Result included are no larger than
pivot, and all elements in the other “high” portion (after position Result) of the array
are no smaller than pivot. Formally, the postcondition is:
0 ≤ Result ≤ n
perm (a, old a)
a[1..Result] ≤ pivot
a[Result + 1..n] ≥ pivot.
In the special case where all elements in a are greater than or equal to pivot, Result
will be zero, corresponding to the “low” portion of the array being empty.
Quick sort works by partitioning an array and then recursively partitioning each of
the two portions of the partition. The choice of pivot at every recursive call is crucial to
guarantee a good performance of quick sort. Its correctness, however, relies solely on
the correctness of partition, not on the choice of pivot. Hence, the focus of this section
is on partition alone.
The bulk of the loop invariant follows from the last three clauses of the postcondition.
perm (a, old a) appears unchanged in the essential invariant, denoting the fact that
the whole algorithm does not change a’s elements but only rearranges them. The
clauses comparing a’s slices to pivot determine the rest of the essential invariant, once
we modify them by introducing loop variables low and high decoupling and relaxing
“constant” Result:
perm (a, old a)
a [1..low − 1] ≤ pivot
a [high + 1..n] ≥ pivot.
The formula low = high—removed when decoupling—becomes the main loop’s exit
condition. Finally, a similar variable introduction applied twice to the postcondition
0 ≤ Result ≤ n suggests the bounding invariant clause
1 ≤ low ≤ high ≤ n.
The slice comparison a [1..low − 1] ≤ pivot also includes aging of variable low. This
makes the invariant clauses fully symmetric and suggests a matching implementation
with two inner loops nested inside an overall outer loop. The outer loop starts with
low = 1 and high = n and terminates, with low = high, when the whole array has been
processed. The first inner loop increments low until it points to an element that is larger
than pivot, and hence is in the wrong portion of the array. Symmetrically, the outer loop
decrements high until it points to an element smaller than pivot. After low and high are
set by the inner loops, the outer loop swaps the corresponding elements, thus making
progress toward partitioning the array. Figure 11 shows the resulting implementation.
The closing conditional in the main routine’s body ensures that Result points to an

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:24 C. A. Furia et al.

Fig. 11. Quick sort: partitioning.

element no greater than pivot; this is not enforced by the loop, whose invariant leaves
the value of a [low] unconstrained. In particular, in the special case of all elements
being no less than pivot, low and Result are set to zero after the loop.
In the correctness proof, it is useful to discuss the cases a [low] < pivot and a [low] ≥
pivot separately when proving consecution. In the former case, we combine a [1..low −
1] ≤ pivot and a [low] < pivot to establish the backward substitution a [1..low] ≤ pivot.
In the latter case, we combine low = high, a [high + 1..n] ≥ pivot, and a [low] ≥ pivot
to establish the backward substitution a [low..n] ≥ pivot. The other details of the proof
are straightforward.
4.3.2. Selection Sort. Selection sort is a straightforward sorting algorithm based on a
simple idea: to sort an array, find the smallest element, put it in the first position, and
repeat recursively from the second position on. Pre- and postcondition are the usual
ones for sorting (see Section 4.3) and hence require no further comment.
The first postcondition clause perm (a, old a) is also an essential loop invariant:
perm (a, old a). (20)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:25

If we introduce a variable i to iterate over the array, another essential invariant clause
is derived by writing a in slice form a[1..n] and then by relaxing n into i:
sorted (a [1..i]) (21)
with the bounding clause
1 ≤ i ≤ n, (22)
which ensures that the sorted slice a[1..i] is always nonempty. The final component of
the invariant is also an essential weakening of the postcondition but is less straight-
forward to derive by syntactic mutation. If we split a [1..n] into the concatenation
a [1..i − 1] ◦ a [i..n], we notice that sorted (a [1..i − 1] ◦ a [i..n]) implies
∀k ∈ [i..n] : a [1..i − 1] ≤ a[k] (23)
as a special case. Formula (23) guarantees that the slice a[i..n], which has not been
sorted yet, contains elements that are no smaller than any of those in the sorted slice
a[1..i − 1].
The loop invariants (20)–(22) apply—possibly with minimal changes due to inessen-
tial details in the implementation—for any sorting algorithm that sorts an array se-
quentially, working its way from lower to upper indices. To implement the behavior
specific to selection sort, we introduce an inner loop that finds the minimum element in
the slice a [i..n], which is not yet sorted. To this end, it uses variables j and m: j scans
the slice sequentially starting from position i + 1; m points to the minimum element
found so far. Correspondingly, the inner loop’s postcondition is a[m] ≤ a[i..n], which
induces the essential invariant clause
a [m] ≤ a[i.. j − 1] (24)
specific to the inner loop, by constant relaxation and aging. The outer loop’s invariant
(23) clearly also applies to the inner loop—which does not change i or n—where it
implies that the element in position m is an upper bound on all elements already
sorted:
a [1.. i − 1] ≤ a[m]. (25)
Also specific to the inner loop are more complex bounding invariants relating the values
of i, j, and m to the array bounds:
1 ≤ i < j ≤ n+ 1
i ≤ m < j.
The implementation in Figure 12 follows these invariants. The outer loop’s only task is
then to swap the “minimum” element pointed to by m with the lowest available position
pointed to by i.
The most interesting aspect of the correctness proof is proving consecution of the
outer loop’s invariant clause (21), and in particular that a[i] ≤ a[i + 1]. To this end,
notice that (24) guarantees that a [m] is the minimum of all elements in positions from
i to n, and that (25) is an upper bound on the other elements in positions from 1 to
i − 1. In particular, a[m] ≤ a[i + 1] and a[i − 1] ≤ a[m] hold before the swap on line
30. After the swap, a[i] equals the previous value of a[m]; thus a[i − 1] ≤ a[i] ≤ a[i +
1] holds as required. A similar reasoning proves the inductiveness of the main loop’s
other invariant clause (23).
4.3.3. Insertion Sort. Insertion sort is another suboptimal sorting algorithm that is,
however, simple to present and implement and reasonably efficient on arrays of small
size. As the name suggests, insertion sort hinges on the idea of rearranging elements
in an array by inserting them in their correct positions with respect to the sorting
ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:26 C. A. Furia et al.

Fig. 12. Selection sort.

order; insertion is done by shifting the elements to make room for insertion. Pre- and
postcondition are the usual ones for sorting (see Section 4.3 and the comments in the
previous subsections).
The main loop’s essential invariant is as in selection sort (Section 4.3.2) and other
similar algorithms, as it merely expresses the property that the sorting has progressed
up to position i and has not changed the array content:
sorted (a[1..i]) (26)
perm (a, old a). (27)
This essential invariant goes together with the bounding clause 1 ≤ i ≤ n.
The main loop includes an inner loop, whose invariant captures the specific strategy
of insertion sort. The outer loop’s invariant (27) must be weakened, because the inner
loop overwrites a [i] while progressively shifting to the right elements in the slice
a [1..j]. If a local variable v stores the value of a [i] before entering the inner loop, we
can weaken (27) as:
perm (a [1.. j] ◦ v ◦ a[ j + 2..n], old a), (28)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:27

Fig. 13. Insertion sort.

where “◦” is the concatenation operator; that is, a’s element at position j + 1 is the
current candidate for inserting v—the value temporarily removed. After the inner loop
terminates, the outer loop will put v back into the array at position j + 1 (line 28 in
Figure 13), thus restoring the stronger invariant (27) (and establishing inductiveness
for it).
The clause (26), crucial for the correctness argument, is also weakened in the inner
loop. First, we “age” i by replacing it with i − 1; this corresponds to the fact that the
outer loop increments i at the beginning, and will then re-establish (26) only at the end
of each iteration. Therefore, the inner loop can only assume the weaker invariant:
sorted (a [1..i − 1]) (29)
that is not invalidated by shifting (which only temporarily duplicates elements). Shift-
ing has, however, another effect: since the slice a[j + 1..i] contains elements shifted up
from the sorted portion, the slice a[j + 1..i] is itself sorted, thus the essential invariant:
sorted (a [ j + 1..i]). (30)
We can derive the pair of invariants (29)–(30) from the inner loop’s postcondition (26):
write a [1..i] as a [1..i − 1] ◦ a[i..i]; weaken the formula sorted (a [1..i − 1] ◦ a[i..i]) into
the conjunction of sorted ( a [1..i − 1]) and sorted (a[i..i]); and replace one occurrence of

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:28 C. A. Furia et al.

constant i in the second conjunct by a fresh variable j and age to derive sorted (a [j +
1..i]).
Finally, there is another essential invariant, specific to the inner loop. Since the
loop’s goal is to find a position, pointed to by j + 1, before i where v can be inserted, its
postcondition is:
v ≤ a [ j + 1..i], (31)
which is also a suitable loop invariant, combined with a bounding clause that constrains
j and i:
0 ≤ j < i ≤ n. (32)
Overall, clauses (28)–(32) are the inner loop invariant, and Figure 13 shows the match-
ing implementation.
As usual for this kind of algorithm, the crux of the correctness argument is proving
that the outer loop’s essential invariant is inductive, based on the inner loop’s. The
formal proof uses the following informal argument. Formulas (29) and (31) imply that
inserting v at j + 1 does not break the sortedness of the slice a [1..j + 1]. Furthermore,
(30) guarantees that the elements in the “upper” slice a [j + 1..i] are also sorted with
a [j] ≤ a[j + 1] ≤ a[j + 2]. (The detailed argument would discuss the cases j = 0,
0 < j < i − 1, and j = i − 1.) In all, the whole slice a [1..i] is sorted, as required by (26).
4.3.4. Bubble Sort (Basic). As a sorting method, bubble sort is known not for its perfor-
mance but for its simplicity [Knuth 2011, Vol. 3, Sec. 5.2.2]. It relies on the notion of
inversion: a pair of elements that are not ordered, that is, such that the first is greater
than the second. The straightforward observation that an array is sorted if and only
if it has no inversions suggests to sort an array by iteratively removing all inversions.
Let us present invariants that match such a high-level strategy, deriving them from
the postcondition (which is the same as the other sorting algorithms of this section).
The postcondition perm (a, old a) that a’s elements be not changed is also an invariant
of the two nested loops used in bubble sort. The other postcondition sorted (a) is instead
weakened, but in a way different than in other sorting algorithms seen before. We
introduce a Boolean flag swapped, which records if there is some inversion that has
been removed by swapping a pair of elements. When swapped is false after a complete
scan of the array a, no inversions have been found, and hence a is sorted. Therefore,
we use ¬ swapped as exit condition of the main loop, and the weakened postcondition
¬ swapped =⇒ sorted (a) (33)
as its essential loop invariant.
The inner loop performs a scan of the input array that compares all pairs of adjacent
elements and swaps them when they are inverted. Since the scan proceeds linearly
from the first element to the last one, we get an essential invariant for the inner loop
by replacing n by i in (33) written in slice form:
¬ swapped =⇒ sorted (a[1..i]). (34)
The usual bounding invariant 1 ≤ i ≤ n and the outer loop’s invariant clause perm (a,
old a) complete the inner loop invariant.
The implementation is now straightforward to write as in Figure 14. The inner loop,
in particular, sets swapped to True whenever it finds some inversion while scanning.
This signals that more scans are needed before the array is certainly sorted.
Verifying the correctness of the annotated program in Figure 14 is easy, because the
essential loop invariants (33) and (34) are trivially true in all iterations where swapped
is set to True. On the other hand, this style of specification makes the termination

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:29

Fig. 14. Bubble sort (basic version).

argument more involved: the outer loop’s variant (line 28 in Figure 14) must explicitly
refer to the number of inversions left in a, which are decreased by complete executions
of the inner loop.
4.3.5. Bubble Sort (Improved). The inner loop in the basic version of bubble sort—
presented in Section 4.3.4—always performs a complete scan of the n-element array
a. This is often redundant, because swapping adjacent inverted elements guarantees
that the largest misplaced element is sorted after each iteration. Namely, the largest
element reaches the rightmost position after the first iteration, the second-largest one
reaches the penultimate position after the second iteration, and so on. This section
describes an implementation of bubble sort that takes advantage of this observation to
improve the running time.
The improved version still uses two nested loops. The outer loop’s essential invariant
has two clauses:
sorted (a [i..n]) (35)
is a weakening of the postcondition that encodes the knowledge that the “upper” part
of array a is sorted, and
i < n =⇒ a[1..i] ≤ a[i + 1] (36)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:30 C. A. Furia et al.

Fig. 15. Bubble sort (improved version).

specifies that the elements in the unsorted slice a[1..i] are no larger than the first
“sorted” element a[i + 1]. The expression a[1..i] ≤ a[i + 1] is a mutation (constant
relaxation and aging) of a[1..n] ≤ a[n], which is, in turn, a domain property following
from the postcondition. Variable i is now used in the outer loop to mark the portion still
to be sorted; correspondingly, (36) is well defined only when i < n, and the bounding
invariant clause 1 ≤ i ≤ n is also part of the outer loop’s specification.
Continuing with the same logic, the inner loop’s postcondition:
a[1..i] ≤ a[i] (37)
states that the largest element in the slice a [1..i] has been moved to the highest
position. Constant relaxation, replacing i (not changed by the inner loop) with a fresh
variable j, yields a new essential component of the inner loop’s invariant:
a[1.. j] ≤ a[ j]. (38)
The outer loop’s invariant and the bounding clause 1 ≤ j ≤ i complete the specification
of the inner loop. Figure 15 displays the corresponding implementation.
The correctness proof follows standard strategies. In particular, the inner loop’s post-
condition (37), that is, the inner loop’s invariant when j = i—implies a[i − 1] ≤ a[i]

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:31

as a special case. This fact combines with the other clause (36) to establish the induc-
tiveness of the main loop’s essential clause:
sorted (a [i..n]).
Finally, proving termination is trivial for this program because each loop has an asso-
ciated iteration variable that is unconditionally incremented or decremented.
4.3.6. Comb Sort. In an attempt to improve performance in critical cases, comb sort
generalizes bubble sort based on the observation that small elements initially stored
in the right-most portion of an array require a large number of iterations to be sorted.
This happens because bubble sort swaps adjacent elements; hence, it takes n scans of
an array of size n just to bring the smallest element from the right-most nth position to
the first one, where it belongs. Comb sort adds the flexibility of swapping nonadjacent
elements, thus allowing for a faster movement of small elements from right to left. A
sequence of nonadjacent equally spaced elements also conveys the image of a comb’s
teeth, hence the name “comb sort.”
Let us make this intuition rigorous and generalize the loop invariants, and the
implementation, of the basic bubble sort algorithm described in Section 4.3.4. Comb
sort is also based on swapping elements, therefore the—now well-known—invariant
perm (a, old a) also applies to its two nested loops. To adapt the other loop invariant
(33), we need a generalization of the predicate sorted that fits the behavior of comb
sort. Predicate gap_sorted (a, d), defined as:
gap sorted (a, d) ⇐⇒ ∀k ∈ [a.lower..a.upper − d] : a[k] ≤ a[k + d]
holds for arrays a such that the subsequence of d-spaced elements is sorted. Notice
that, for d = 1, gap_sorted reduces to sorted:
gap sorted (a, 1) ⇐⇒ sorted (a).
This fact will be used to prove the postcondition from the loop invariant upon
termination.
With this new piece of domain theory, we can easily generalize the essential and
bounding invariants of Figure 14 to comb sort. The outer loop considers decreasing
gaps; if variable gap stores the current value, the bounding invariant
1 ≤ gap ≤ n
defines its variability range. Precisely, the main loop starts with with gap = n and
terminates with gap = 1, satisfying the essential invariant:
¬swapped =⇒ gap sorted (a, gap). (39)
The correctness of comb sort does not depend on how gap is decreased, as long as it
eventually reaches 1; if gap is initialized to 1, comb sort behaves exactly as bubble sort.
In practice, it is customary to divide gap by some chosen parameter c at every iteration
of the main loop.
Let us now consider the inner loop, which compares and swaps the subsequence of
d-spaced elements. The bubble sort invariant (34) generalizes to:
¬swapped =⇒ gap sorted (a [1..i − 1 + gap], gap) (40)
and its matching bounding invariant is:
1 ≤ i < i + gap ≤ n + 1
so that when i = n + 1 + gap the inner loop terminates and (40) is equivalent to (39).
This invariant follows from constant relaxation and aging; the substituted expression

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:32 C. A. Furia et al.

Fig. 16. Comb sort.

i − 1 + gap is more involved, to accommodate how i is used and updated in the inner
loop, but is otherwise semantically straightforward.
The complete implementation is shown in Figure 16. The correctness argument is
exactly as for bubble sort in Section 4.3.4 but exploits the properties of the generalized
predicate gap_sorted instead of the simpler sorted.

4.4. Dynamic Programming

Dynamic programming is an algorithmic technique used to compute functions that
have a natural recursive definition. Dynamic programming algorithms construct solu-
tions iteratively and store the intermediate results, so that the solution to larger in-
stances can reuse the previously computed solutions for smaller instances. This section
presents a few examples of problems that lend themselves to dynamic programming
solutions.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:33

4.4.1. Unbounded Knapsack Problem with Integer Weights. We have an unlimited collection
of items of n different types. An item of type k, for k = 1, . . . , n, has weight w[k] and
value v[k]. The unbounded knapsack problem asks what is the maximum overall value
that one can carry in a knapsack whose weight limit is a given weight. The attribute
“unbounded” refers to the fact that we can pick as many objects of any type as we
want: the only limit is given by the input value of weight, and by the constraint that
we cannot store fractions of an item—either we pick it or we don’t.
Any vector s of n nonnegative integers defines a selection of items whose overall
weight is given by the scalar product:

s·w = s[k]w[k]
1≤k≤n

and whose overall value is similarly given by the scalar product s·v. Using this notation,
we introduce the domain theoretical function max_knapsack, which defines the solution
of the knapsack problem given a weight limit b and items of n types with weight and
value given by the vectors w and v:
∃s ∈ Nn : s · w ≤ b ∧ s · v = κ
max knapsack (b, v, w, n) = κ ⇐⇒ ∧ ,
∀t ∈ Nn : t · w ≤ b =⇒ t · v ≤ κ

that is, the largest value achievable with the given limit. Whenever weights w, values v,
and number n of item types are clear from the context, we will abbreviate max_knapsack
(b, v, w, n) by just K(b).
The unbounded knapsack problem is NP-complete [Garey and Johnson 1979; Kellerer
et al. 2004]. It is, however, weakly NP-complete [Papadimitriou 1993], and in fact it
has a nice solution with pseudo-polynomial complexity based on a recurrence relation,
which suggests a straightforward dynamic programming algorithm. The recurrence
relation defines the value of K(b) based on the values K(b ) for b < b.
The base case is for b = 0. If we assume, without loss of generality, that no item has
null weight, it is clear that we cannot store anything in the knapsack without adding
some weight, and hence the maximum value attainable with a weight limit zero is also
zero: K(0) = 0. Let now b be a generic weight limit greater than zero. To determine the
value of K(b), we make a series of attempts as follows. First, we select some item of
type k, such that w[k] ≤ b. Then, we recursively consider the best selection of items for
a weight limit of b − w[k], and we set up a new selection by adding one item of type k
to it. The new configuration has weight no greater than b − w[k] + w[k] = b and value
v[k] + K(b − w[k]),
which is, by inductive hypothesis, the largest achievable by adding an element of type
k. Correspondingly, the recurrence relation defines K(b) as the maximum among all
values achievable by adding an object of some type:

0  b=0
K(b) = (41)
max{v[k] + K(b − w[k])  k ∈ [1..n] and 0 ≤ w[k] ≤ b} b > 0.

The dynamic programming solution to the knapsack problem presented in this sec-
tion computes the recursive definition (41) for increasing values of b. It inputs arrays v
and w (storing the values and weights of all elements), the number n of element types,
and the weight-bound weight. The precondition requires that weight be nonnegative,
that all element weights w be positive, and that the arrays v and w be indexed from 1

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:34 C. A. Furia et al.

to n:
weight ≥ 0
w>0
v.lower = w.lower = 1
v.upper = w.upper = n.
The last two clauses are merely for notational convenience and could be dropped. The
postcondition states that the routine returns the value K(weight) or, with more precise
notation:
Result = max knapsack (weight, v, w, n).
The main loop starts with b = 0 and continues with unit increments until b = weight;
each iteration stores the value of K(b) in the local array m, so that m [weight] will contain
the final result upon termination. Correspondingly, the main loop’s essential invariant
follows by constant relaxation:
—Variable m [weight] replaces constant Result, thus connecting m to the returned
result.
—The range of variables [1..b] replaces constant weight, thus expressing the loop’s
incremental progress.
The loop invariant is thus:
∀y ∈ [0..b] : m[y] = max knapsack (y, v, w, n), (42)
which goes together with the bounding clause
0 ≤ b ≤ weight
that qualifies b’s variability domain. With a slight abuse of notation, we concisely write
(42), and similar expressions, as:
m[0..b] = max knapsack ([0..b], v, w, n). (43)
The inner loop computes the maximum of definition (41) iteratively, for all element
types j, where 1 ≤ j ≤ n. To derive its essential invariant, we first consider its postcon-
dition (similarly as the analysis of selection sort in Section 4.3.2). Since the inner loop
terminates at the end of the outer loop’s body, the inner’s postcondition is the outer’s
invariant (43). Let us rewrite it by highlighting the value m[b] computed in the latest
iteration:
m[0..b − 1] = max knapsack ([0..b − 1], v, w, n) (44)
m[b] = best value (b, v, w, n, n). (45)
Function best_value is part of the domain theory for knapsack, and it expresses the
“best” value that can be achieved given a weight limit of b, j ≤ n element types, and
assuming that the values K(b ) for lower-weight limits b < b are known:

best value (b, v, w, j, n) = max{v[k] + K(b − w[k])  k ∈ [1.. j] and 0 ≤ w[k] ≤ b}.
If we substitute variable j for constant n in (45), expressing the fact that the inner loop
tries one element type at a time, we get the inner loop essential invariant:
m[0..b − 1] = max knapsack ([0..b − 1], v, w, n)
m[b] = best value (b, v, w, j, m).
The obvious bounding invariants 0 ≤ b ≤ weight and 0 ≤ j ≤ n complete the inner loop’s
specification. Figure 17 shows the corresponding implementation.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:35

Fig. 17. Unbounded knapsack problem with integer weights.

The correctness proof reverses the construction we highlighted following the usual
patterns seen in this section. In particular, notice that:
—When j = n the inner loop terminates, thus establishing (44) and (45).
—(44) and (45) imply (43) because the recursive definition (41) for some b only depends
on the previous values for b < b and (44) guarantees that m stores those values.
4.4.2. Levenshtein Distance. The Levenshtein distance of two sequences s and t is the
minimum number of elementary edit operations (deletion, addition, or substitution of
one element in either sequence) necessary to turn s into t. The distance has a natural
recursive definition:
⎧
⎪
⎪ 0 m= n= 0
⎪
⎪m m > 0, n = 0
⎪
⎪
⎪
⎨n n > 0, m = 0
distance(s, t) = distance(s[1..m − 1], t[1..n − 1]) m > 0, n > 0, s[m] = t[n]
⎪
⎪ −
⎪
⎪ distance(s[1..m 1], t),
⎪
⎪ 1 + min distance(s, t[1..n − 1]), m > 0, n > 0, s[m] = t[n],
⎪
⎩
distance(s[1..m − 1], t[1..n − 1])

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:36 C. A. Furia et al.

where m and n, respectively, denote s’s and t’s length (written s.count and t.count when
s and t are arrays). The first three cases of the definition are trivial and correspond
to when s, t, or both are empty: the only way to get a nonempty string from an empty
one is by adding all the former’s elements. If s and t are both nonempty and their last
elements coincide, then the same number of operations as for the shorter sequences
s[1..m − 1] and t[1..n − 1] (which omit the last elements) are needed. Finally, if s’s and
t’s last elements differ, there are three options: (1) delete s[m] and then edit s[1..m− 1]
into t; (2) edit s into t[1..n− 1] and then add t[n] to the result; (3) substitute t[n] for s[m]
and then edit the rest s[1..m − 1] into t[1..n − 1]. Whichever of the options (1), (2), and
(3) leads to the minimal number of edit operations is the Levenshtein distance.
It is natural to use a dynamic programming algorithm to compute the Levenshtein
distance according to its recursive definition. The overall specification, implementa-
tion, and corresponding proofs are along the same lines as the knapsack problem of
Section 4.4.1; therefore, we briefly present only the most important details. The post-
condition is simply
Result = distance (s, t).
The implementation incrementally builds a bidimensional matrix d of distances
such that the element d[i, j] stores the Levenshtein distance of the sequences s[1..i]
and t[1.. j]. Correspondingly, there are two nested loops: the outer loop iterates over
rows of d, and the inner loop iterates over each column of d. Their essential invariants
express, through quantification, the partial progress achieved after each iteration:
∀h ∈ [0..i − 1], ∀k ∈ [0..n] : d[h, k] = distance(s[1..h], t[1..k])
∀h ∈ [0..i − 1], ∀k ∈ [0.. j − 1] : d[h, k] = distance(s[1..h], t[1..k]).
The standard bounding invariants on the loop variables i and j complete the
specification.
Figure 18 shows the implementation, which uses the compact across notation for
loops, similar to “for” loops in other languages. The syntax
across [a..b] invariant I as k loop B end
is simply a shorthand for:
from k := a invariant I until k = b + 1 loop B; k := k + 1 end
For brevity, Figure 18 omits the obvious loop invariants of the initialization loops at
lines 11 and 12.
4.5. Computational Geometry: Rotating Calipers
The diameter of a polygon is its maximum width, that is, the maximum distance
between any pair of its points. For a convex polygon, it is clear that a pair of vertices
determine the diameter (such as vertices p3 and p7 in Figure 19). Shamos showed [1978]
that it is not necessary to check all O(n2 ) pairs of vertices: his algorithm, described
later, runs in time O(n). The correctness of the algorithm rests on the notions of lines
of support and antipodal points. A line of support is analogous to a tangent: a line of
support of a convex polygon p is a line that intersects p such that the interior of p lies
entirely to one side of the line. An antipodal pair is then any pair of p’s vertices that
lie on two parallel lines of support. It is a geometric property that an antipodal pair
determines the diameter of any polygon p, and that a convex polygon with n vertices
has O(n) antipodal pairs. Figure 19(a), for example, shows two parallel lines of support
that identify the antipodal pair ( p1 , p5 ).
Shamos’s algorithm efficiently enumerates all antipodal pairs by rotating two lines
of support while maintaining them parallel. After a complete rotation around the
polygon, they have touched all antipodal pairs, and hence the algorithm can terminate.
ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:37

Fig. 18. Levenshtein distance.

Fig. 19. The rotating calipers algorithm illustrated.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:38 C. A. Furia et al.

Fig. 20. Diameter of a polygon with rotating calipers.

Observing that two parallel support lines resemble the two jaws of a caliper, Toussaint
[1983] suggested the name “rotating calipers” to describe Shamos’s technique.
A presentation of the rotating calipers algorithm and a proof of its correctness with
the same level of detail as the algorithms in the previous sections would require the
development of a complex domain theory for geometric entities and of the corresponding
implementation primitives. Such a level of detail is beyond the scope of this article;
instead, we outline the essential traits of the specification and give a description of the
algorithm in pseudo-code in Figure 20.6

6 The algorithm in Figure 20 is slightly simplified, as it does not deal explicitly with the special case where a
line initially coincides with an edge: then, the minimum angle is zero, and hence the next vertex not on the

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:39

The algorithm inputs a list p of at least three points such that it represents a
convex polygon (precondition on line 3) and returns the value of the polygon’s diameter
(postcondition on line 41).
It starts by adjusting the two parallel support lines on the two vertices with the
maximum difference in y coordinate, such as p1 and p5 in Figure 19(a). Then, it enters
a loop that computes the angle between each line and the next vertex on the polygon
and rotates both jaws by the minimum of such angles. At each iteration, it compares
the distance between the new antipodal pair and the currently stored maximum (in
Result) and updates the latter if necessary. In the example of Figure 19(a), jaw_a
determines the smallest angle, and hence both jaws are rotated by angle_a in Figure
19(b). Such an informal description suggests the obvious bounding invariants that
the two jaws are maintained parallel (line 18), Result varies between some initial
value and the final value diameter (p) (line 19), and the total rotation of the calipers
is between zero and 180 + 180 (line 20). The essential invariant is, however, harder
to state formally, because it involves a subset of the antipodal pairs reached by the
calipers. A semiformal presentation is:
  p1 , p2 ∈ p ∧ 


Result = max | p1 , p2 | reached ( p1 , total rotation) ∧
reached ( p2 , total rotation)
whose intended meaning is that Result stores the maximum distance between all
points p1 , p2 among p’s vertices that can be reached with a rotation of up to to-
tal_rotation degrees from the initial calipers’ horizontal positions.

4.6. Algorithms on Data Structures

Many data structures are designed around specific operations, which can be performed
efficiently by virtue of the characterizing properties of the structures. This section
presents linked lists and binary search trees and algorithms for some of such op-
erations. The presentation of their invariants clarifies the connection between data-
structure properties and the algorithms’ correct design.
4.6.1. List Reversal. Consider a list of elements of generic type G implemented as a
linked list: each element’s attribute next stores a reference to the next element in the
list, and the last elements’s next attribute is Void. This section discusses the classic
algorithm that reverses a linked list iteratively.
We introduce a specification that abstracts some implementation details by means
of a suitable domain theory. If list is a variable of type LIST [G]—that is, a reference
to the first element—we lift the semantics of list in assertions to denote the sequence
of elements found by following the chain of references until Void. This interpretation
defines finite sequences only if list’s reference sequence is acyclic, which we write acyclic
(list). Thus, the precondition of routine reverse is simply
acyclic (list),
where list is the input linked list.
For a sequence s = s1 s2 . . . sn of elements of length n ≥ 0, its reversal rev (s) is
inductively defined as:
 n= 0
rev(s) = (46)
rev(s2 . . . sn) ◦ s1 n > 0,

line should be considered. This problem can be avoided by adjusting the initialization to avoid that a line
coincides with an edge.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:40 C. A. Furia et al.

Fig. 21. One iteration of the loop in routine reverse.

where  denotes the empty sequence and “◦” is the concatenation operator. With this
notation, reverse’s postcondition is:
list = rev(old list)
with the matching property that list is still acyclic.
The domain theory for lists makes it possible to derive the loop invariant with
the usual techniques. Let us introduce a local variable reversed, which will store the
iteratively constructed reversed list. More precisely, every iteration of the loop removes
one element from the beginning of list and moves it to the beginning of reversed, until
all elements have been moved. When the loop terminates:
—list points to an empty list;
—reversed points to the reversal of old list.
Therefore, the routine concludes by assigning reversed to overwrite list. Backward
substitution yields the loop’s postcondition from the routine’s:
reversed = rev(old list). (47)
Using (46) for empty lists, we can equivalently write (47) as:
rev(list) ◦ reversed = rev(old list), (48)
which is the essential loop invariant, whereas list = Void is the exit condition. The
other component of the loop invariant is the constraint that list and reversed be acyclic,
also by mutation of the postcondition.
Figure 22 shows the standard implementation. Figure 21 pictures instead a graphical
representation of reverse’s behavior: Figure 21(a) shows the state of the lists in the
middle of the computation, and Figure 21(b) shows how the state changes after one
iteration, in a way that the invariant is preserved.
The correctness proof relies on some straightforward properties of the rev and ◦
functions. Initiation follows from the property that s ◦  = s. Consecution relies on the
definition (46) for n > 0, so that:
rev (list.next) ◦ list.first = rev (list).

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:41

Fig. 22. Reversal of a linked list.

Proving the preservation of acyclicity relies on the two properties:

acyclic(s1 s2 . . . sn) =⇒ acyclic(s2 . . . sn)
|s1 | = 1 ∧ acyclic(r) =⇒ acyclic(s1 ◦ r).
4.6.2. Binary Search Trees. Each node in a binary tree has at most two children, con-
ventionally called left and right. Binary search trees are a special kind of binary trees
whose nodes store values from some totally ordered domain T and are arranged re-
flecting the relative order of their values; namely, if t is a binary search tree and n ∈ t
is one of its nodes with value v, all nodes in n’s left subtree store values less than or
equal to v, and all nodes in n’s right subtree store values greater than or equal to v. We
express this characterizing property using domain theory notation as:
s ∈ t[n.left] =⇒ s.value ≤ n.value
(49)
s ∈ t[n.right] =⇒ s.value ≥ n.value,
where t[n] denotes t’s subtree rooted at node n. This property underpins the correctness
of algorithms for operations such as searching, inserting, and removing nodes in binary
search trees that run in time linearly in a tree’s height; for trees whose nodes are
properly balanced, the height is logarithmic in the number of nodes, and hence the
operations can be performed efficiently. We now illustrate two of these algorithms with
their invariants.
Consider searching for a node with value key in a binary search tree t. If t.values
denotes the set of values stored in t, the specification of this operation consists of the
postcondition
key ∈ t.values =⇒ Result ∈ t ∧ key = Result.value (50)
key ∈ t.values =⇒ Result = Void, (51)
where Void is returned if no node has value key. For simplicity, we only consider
nonempty trees—handling the special case of an empty tree is straightforward.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:42 C. A. Furia et al.

Fig. 23. Search in a binary search tree.

We can obtain the essential invariant by weakening both conjuncts in (50) based on
two immediate properties of trees. First, Result ∈ t implies Result = Void, because
no valid node is Void. Second, a node’s value belongs to the set of values of the subtree
rooted at the node:
n.value ∈ t[n].values
for n ∈ t. Thus, the following formula is a weakening of (50):
key ∈ t.values =⇒ Result = Void ∧ key ∈ t[Result].values, (52)
which works as an essential loop invariant for binary-search-tree search.
Search works by moving Result to the left or right subtree—according to (49)—until
a value key is found or the subtree to be explored is empty. This corresponds to the
disjunctive exit condition Result = Void ∨ Result.value = key and to the bounding
invariant Result = Void =⇒ Result ∈ t: we are within the tree until we hit Void.
Figure 23 shows the corresponding implementation.
Initiation follows from the precondition and from the identity t[t.root] = t. Consecu-
tion relies on the following domain property, which in turn follows from (49):
n ∈ t ∧ v ∈ t[n].values ∧ v < n.value =⇒ n.left = Void ∧ v ∈ t[n.left].values
n ∈ t ∧ v ∈ t[n].values ∧ v > n.value =⇒ n.right = Void ∧ v ∈ t[n.right].values.

The ordering property (49) entails that the leftmost node in a (nonempty) tree t—that
is, the first node without left child reached by always going left from the root—stores
the minimum of all node values. This property, expressible using the domain theory
function leftmost as:
min(t.values) = leftmost(t).value, (53)
leads to an algorithm to determine the node with minimum value in a binary search
tree, whose postcondition is thus:
Result = leftmost(t) (54)
Result.value = min(t.values). (55)

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:43

Fig. 24. Minimum in a binary search tree.

The algorithm only has to establish (54), which then implies (55) combined with the
property (53). In fact, the algorithm is oblivious of (49) and operates solely based on
structural properties of binary trees; (55) follows as an afterthought.
Duplicating the right-hand side of (54), writing t in the equivalent form t[t.root], and
applying constant relaxation to t.root yields the essential invariant
leftmost (t [Result]) = leftmost (t) (56)
with the bounding invariant Result ∈ t that we remain inside the tree t. These invari-
ants capture the procedure informally highlighted earlier: walk down the left children
until you reach the leftmost node. The corresponding implementation is in Figure 24.
Initiation follows by trivial identities. Consecution relies on a structural property of
the leftmost node in any binary tree:
n ∈ t ∧ n.left = Void =⇒ n.left ∈ t ∧ leftmost(t[n]) = leftmost(t[n.left]).

4.7. Fixpoint Algorithms: PageRank

PageRank is a measure of the popularity of nodes in a network, used by the Google
Internet search engine. The basic idea is that the PageRank score of a node is higher
the more nodes link to it (multiple links from the same page or self-links are ignored).
More precisely, the PageRank score is the probability that a random visit on the graph
(with uniform probability on the outgoing links) reaches it at some point in time.
The score also takes into account a dampening factor, which limits the number of
consecutive links followed in a random visit. If the graph is modeled by an adjacency
matrix (modified to take into account multiple and self-links, and to make sure that
there are no sink nodes without outgoing edges), the PageRank scores are the entries
of the dominant eigenvector of the matrix.
In our presentation, the algorithm does not deal directly with the adjacency matrix
but inputs information about the graph through arguments reaching and outbound.
The former is an array of sets of nodes: reaching [k] denotes the set of nodes that
directly link to node k. The other argument outbound [k] denotes instead the number of
outbound links (to different nodes) originating in node k. The Result is a vector of n real
numbers, encoded as an array, where n is the number of nodes. If eigenvector denotes the
dominant eigenvector (also of length n) of the adjacency matrix (defined implicitly), the
postcondition states that the algorithm computes the dominant eigenvector to within

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:44 C. A. Furia et al.

Fig. 25. PageRank fixpoint algorithm.

precision  (another input):

|eigenvector − Result| < . (57)
That is, Result [k] is the rank of node k to within overall precision .
The algorithm computes the PageRank score iteratively: it starts assuming a uniform
probability on the n nodes, and then it updates it until convergence to a fixpoint. Before
every iteration, the algorithm saves the previous values of Result as old_rank, so
that it can evaluate the progress made after the iteration by comparing the sum diff
of all pairwise absolute differences, one for each node, between the scores saved in
old_rank and the newly computed scores available in Result. Correspondingly, the
main loop’s essential invariant is postcondition (57) with diff substituted for . diff
gets smaller with every iteration of the main loop, until it becomes less than , the
main loop terminates, and the postcondition holds. The connection between the main
loop invariants and the postcondition is thus straightforward.
Figure 25 shows an implementation of this algorithm. The two across loops nested
within the main loop update the PageRank scores in Result. Every iteration of the
outer across loop updates the value of Result [i] for node i as:
(1 − dampening) old rank[ j]
+ dampening · . (58)
n j
outbound[ j]

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:45

The inner loop computes the sum in (58) for j ranging over the set reaching [i] of nodes
that directly reach i. The invariants of the across loops express the progress in the
computation of (58); we do not write them down explicitly as they are not particularly
insightful from the perspective of connecting postconditions and loop invariants.

5. RELATED WORK: AUTOMATIC INVARIANT INFERENCE

The amount of research work on the automated inference of invariants is substantial
and spread over more than three decades; this reflects the cardinal role that invariants
play in the formal analysis and verification of programs. This section outlines a few
fundamental approaches that emerged, without any pretense of being exhaustive. A
natural classification of the methods to infer invariants is between static and dynamic.
Static methods (Section 5.1) use only the program text, whereas dynamic methods
(Section 5.2) summarize the properties of many program executions with different
inputs.
Program construction. In the introductory sections, we already mentioned classi-
cal formal methods for program construction [Dijkstra 1976; Gries 1981; Meyer 1980;
Morgan 1994] on which this survey article is based. In particular, the connection be-
tween a loop’s postcondition and its invariant buttresses the classic methods of pro-
gram construction; this survey article has demonstrated it on a variety of examples.
In previous work [Furia and Meyer 2010], we developed gin-pink, a tool that practi-
cally exploits the connection between postconditions and invariants. Given a program
annotated with postconditions, gin-pink systematically generates mutations based on
the heuristics of Section 3.2 and then uses the Boogie program verifier [Leino 2008]
to check which mutations are correct invariants. The gin-pink approach borrows ideas
from both static and dynamic methods for invariant inference: it is only based on the
program text (and specification) as the former, but it generates “candidate” invariants
to be checked—like dynamic methods do.
Reasoning about domain theories. To bridge the gap between the levels of abstrac-
tion of domain theories and of their underlying atomic assertions (see Section 2), one
needs to reason about first- or even higher-logic formulas often involving interpreted
theories such as arithmetic. The research in this area of automated theorem proving is
huge; interested readers are referred to the many reference publications on the topic
[Robinson and Voronkov 2001; Buchberger 2006; Bradley and Manna 2007; Kroening
and Strichman 2008].

5.1. Static Methods

Historically, the earliest methods for invariant inference were static as in the pioneering
work of Karr [1976]. Abstract interpretation and the constraint-based approach are the
two most widespread frameworks for static invariant inference (see also Bradley and
Manna [2007, Chap. 12]). Jhala and Majumdar [2009] provide an overview of the most
important static techniques and discuss how they are applied in combination with
different problems of program verification.
Abstract interpretation is a symbolic execution of programs over abstract domains
that overapproximates the semantics of loop iteration. Since the seminal work by
Cousot and Cousot [1977], the technique has been updated and extended to deal with
features of modern programming languages such as object orientation and heap mem-
ory management (e.g., Logozzo [2004] and Chang and Leino [2005]). One of the main
successes of abstract interpretation has been the development of sound but incom-
plete tools [Blanchet et al. 2003] that can verify the absence of simple and common
programming errors such as division by zero or void dereferencing.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:46 C. A. Furia et al.

Constraint-based techniques rely on sophisticated decision procedures over nontriv-

ial mathematical domains (such as polynomials or convex polyhedra) to represent
concisely the semantics of loops with respect to certain template properties.
Static methods are sound and often complete with respect to the class of invariants
that they can infer. Soundness and completeness are achieved by leveraging the de-
cidability of the underlying mathematical domains they represent; this implies that
the extension of these techniques to new classes of properties is often limited by unde-
cidability. State-of-the-art static techniques can infer invariants in the form of mathe-
matical domains such as linear inequalities [Cousot and Halbwachs 1978; Colón et al.
2003], polynomials [Sankaranarayanan et al. 2004; Rodrı́guez-Carbonell and Kapur
2007], restricted properties of arrays [Bradley et al. 2006; Bozga et al. 2009; Henzinger
et al. 2010], and linear arithmetic with uninterpreted functions [Beyer et al. 2007].
Following Section 3.1, the loop invariants that static techniques can easily infer
are often a form of “bounding” invariant. This suggests that the specialized static
techniques for loop invariant inference discussed in this section and the idea of deriving
the loop invariant from the postcondition, demonstrated in the rest of the article,
can be fruitfully combined: the former can easily provide bounding loop invariants,
whereas the latter can suggest the “essential” components that directly connect to the
postcondition.
To our knowledge, there are only a few approaches to static invariant inference that
take advantage of existing annotations [Păsăreanu and Visser 2004; Janota 2007; de
Caso et al. 2009; Lahiri et al. 2009; Kovács and Voronkov 2009]. Janota [2007] relies
on user-provided assertions nested within loop bodies and tries to check whether they
hold as invariants of the loop. The approach has been evaluated only on a limited
number of straightforward examples. De Caso et al. [2009] briefly discuss deriving
the invariant of a “for” loop from its postcondition within a framework for reasoning
about programs written in a specialized programming language. Lahiri et al. [2009]
also leverage specifications to derive intermediate assertions but focusing on lower-
level and type-like properties of pointers. On the other hand, Păsăreanu and Visser
[2004] derive candidate invariants from postconditions within a framework for symbolic
execution and model checking.
Finally, Kovács and Voronkov [2009] derive complex loop invariants by first encoding
the loop semantics as recurring relations and then instructing a rewrite-based theo-
rem prover to try to remove the dependency on the iterator variables in the relations.
This approach exploits heuristics that, while not guaranteeing completeness, are prac-
tically effective to derive automatically loop invariants with complex quantification—a
scenario that is beyond the capabilities of most other methods.

5.2. Dynamic Methods

Only in the last decade have dynamic techniques been applied to invariant inference.
The Daikon approach of Ernst et al. [2001] showed that dynamic inference is practical
and sprung much derivative work (e.g., Perkings and Ernst [2004], Csallner et al.
[2008], Polikarpova et al. [2009], Ghezzi et al. [2009], Wei et al. [2011a], Nguyen et al.
[2012], and many others). In a nutshell, the dynamic approach consists of testing a large
number of candidate properties against several program runs; the properties that are
not violated in any of the runs are retained as “likely” invariants. This implies that
the inference is not sound but only an “educated guess”: dynamic invariant inference
is to static inference what testing is to program proofs. Nonetheless, just like testing
is quite effective and useful in practice, dynamic invariant inference can work well
if properly implemented. With the latest improvements [Wei et al. 2011b], dynamic
invariant inference can attain soundness of over 99% for the “guessed” invariants.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:47

Among the numerous attempts to improve the effectiveness and flexibility of dynamic
inference, Gupta and Heidepriem [2003] suggest improving the quality of inferred
contracts by using different test suites (based on code coverage and invariant coverage)
and by retaining only the contracts that are inferred with both techniques. Fraser and
Zeller [2011] simplify and improve test cases based on mining recurring usage patterns
in code bases; the simplified tests are easier to understand and focus on common
usage. Other approaches to improve the quality of inferred contracts combine static
and dynamic techniques [Csallner et al. 2008; Tillmann et al. 2006; Wei et al. 2011b].
To date, dynamic invariant inference has been mostly used to infer pre- and postcon-
ditions or intermediate assertions, whereas it has been only rarely applied [Nguyen
et al. 2012] to loop invariant inference. This is probably because dynamic techniques
require a sufficiently varied collection of test cases, and this is more difficult to achieve
for loops—especially if they manipulate complex data structures.

6. LESSONS FROM THE MECHANICAL PROOFS

Our verified Boogie implementations of the algorithms mirror the presentation in
Section 4, as they introduce the predicates, functions, and properties of the domain
theory that are directly used in the specification and in the correctness proofs. The
technology of automatic theorem proving is, however, still in active development and
faces in any case insurmountable theoretical limits of undecidability. As a consequence,
in a few cases we had to introduce explicitly some intermediate domain properties that
were, in principle, logical consequences of other definitions, but that the prover could
not derive without suggestion. Similarly, in other cases we had to guide the proof by
writing down explicitly some of the intermediate steps in a form acceptable to the
verifier.
A lesson of this effort is that the syntactic form in which definitions and properties of
the domain theory are expressed may influence the amount of additional annotations
required for proof automation. The case of the sorting algorithms in Section 4.3 is
instructive. All rely on the definition of predicate sorted as:

∀i ∈ [a.lower..a.upper − 1] : a[i] ≤ a[i + 1], (59)

which compares adjacent elements in positions i and i + 1. Since bubble sort rear-
ranges elements in an array also by swapping adjacent elements, proving it in Boogie
was straightforward, since the prover could figure out how to apply definition (59)
to discharge the various verification conditions—which also feature comparisons of
adjacent elements. Selection sort and insertion sort required substantially more guid-
ance in the form of additional domain theory properties and detailing of intermediate
verification steps, since their logic does not operate directly on adjacent elements,
and hence their verification conditions are syntactically dissimilar to (59). Changing
the definition of sorted into something that relates nonadjacent elements—such as
∀i, j : a.lower ≤ i ≤ j ≤ a.upper =⇒ a[i] ≤ a[ j]—is not sufficient to bridge the semantic
gap between sortedness and other predicates: the logic of selection sort and insertion
sort remains more complex to reason about than that of bubble sort. On the contrary,
comb sort was as easy as bubble sort, because it relies on a generalization of sorted
that directly matches its logic (see Section 4.3.6).
A similar experience occurred for the two dynamic programming algorithms of
Section 4.4. While the implementation of Levenshtein distance closely mirrors the
recursive definition of distance (Section 4.4.2) in computing the minimum, the relation
between specification and implementation is less straightforward for the knapsack
problem (Section 4.4.1), which correspondingly required a more complicated axiomati-
zation for the proof in Boogie.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:48 C. A. Furia et al.

7. CONCLUSIONS AND ASSESSMENT

The concept of loop invariant is, as this review has attempted to show, one of the founda-
tional ideas of software construction. We have seen many examples of the richness and
diversity of practical loop invariants and how they illuminate important algorithms
from many different areas of computer science. We hope that these examples establish
the claim, made at the start of the article, that the invariant is the key to every loop:
to devise a new loop so that it is correct requires summoning the proper invariant; to
understand an existing loop requires understanding its invariant.
Invariants belong to a number of categories, for which this discussion has estab-
lished a classification that we hope readers will find widely applicable and useful in
understanding the loop invariants of algorithms in their own fields. The classifica-
tion is surprisingly simple; perhaps other researchers will find new criteria that have
eluded us.
Descriptions of algorithms in articles and textbooks have increasingly, in recent
years, included loop invariants, at least informally; we hope that the present discussion
will help reinforce this trend and increase the awareness—not only in the research
community but also among software practitioners—of the centrality of loop invariants
in programming.
Informal invariants are good, but being able to express them formally is even better.
Reaching this goal and, more generally, continuing to make invariants ever more main-
stream in software development requires convenient, clear, and expressive notations
for expressing loop invariants and other program assertions. One of the contributions
of this article will be, we hope, to establish the practicality of domain-theory-based
invariants, which express properties at a high level of abstraction, and their superior-
ity over approaches that always revert to the lowest level (suggesting a possible slogan:
“Quantifiers considered harmful”).
Methodological injunctions are necessary, but the history of software practice shows
that they only succeed when supported by effective tools. Many programmers still
find it hard to come up with invariants, and this survey shows that they have some
justifications: even though the basic idea is often clear, coming up with a sound and
complete invariant is an arduous task. Progress in invariant inference, both theoretical
and on the engineering side, remains essential. There is already, as noted, consider-
able work on this topic, often aimed at inferring more general invariant properties
than the inductive loop invariants of the present discussion; but much remains to
be done to bring the tools to a level where they can be integrated in a standard de-
velopment environment and routinely suggest invariants, conveniently and correctly,
whenever a programmer writes a loop. The work mentioned in Section 5 is a step in this
direction.
Another major lesson for us from preparing this survey (and reflecting on how differ-
ent it is from what could have been written on the same topic 30, 20, 10, or even 5 years
ago) came from our success in running essentially all the examples through formal,
mechanized proofs. Verification tools such as Boogie, while still a work in progress,
have now reached a level of quality and practicality that enables them to provide re-
sounding guarantees for work that, in the past, would have remained subject to human
errors.
We hope that others will continue this work, both on the conceptual side, by pro-
viding further insights into the concept of loop invariant, and on the practical side, by
extending the concept to its counterpart for data (i.e., the class invariant) and by broad-
ening our exploration and classification effort to many other important algorithms of
computer science.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:49

ACKNOWLEDGMENTS
This work is clearly based on the insights of Robert Floyd, C.A.R. Hoare, and Edsger Dijkstra in introducing
and developing the notion of loop invariant. The mechanized proofs were made possible by Rustan Leino’s
Boogie tool, a significant advance in turning axiomatic semantics into a practically usable technique. We
are grateful to our former and present ETH colleagues Bernd Schoeller, Nadia Polikarpova, and Julian
Tschannen for pioneering the use of Boogie in our environment.

REFERENCES
Samson Abramsky and Achim Jung. 1994. Domain theory. In Handbook of Logic in Computer Science III, S.
Abramsky, D. M. Gabbay, and T. S. E. Maibaum (Eds.). Oxford University Press.
Dirk Beyer, Thomas A. Henzinger, Rupak Majumdar, and Andrey Rybalchenko. 2007. Invariant synthesis for
combined theories. In Proceedings of the 8th International Conference on Verification, Model Checking,
and Abstract Interpretation (VMCAI’07), Byron Cook and Andreas Podelski (Eds.), Lecture Notes in
Computer Science, vol. 4349. Springer, 378–394.
Bruno Blanchet, Patrick Cousot, Radhia Cousot, Jérôme Feret, Laurent Mauborgne, Antoine Miné, David
Monniaux, and Xavier Rival. 2003. A static analyzer for large safety-critical software. In Proceedings of
the 2003 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’03).
ACM, 196–207.
Joshua Bloch. 2006. Extra, Extra – Read all about it: Nearly all binary searches and mergesorts are broken.
http://googleresearch.blogspot.ch/2006/06/extra-extra-read-all-about-it -nearly.html. (2006).
Marius Bozga, Peter Habermehl, Radu Iosif, Filip Konečný, and Tomáš Vojnar. 2009. Automatic verification
of integer array programs. In Proceedings of the 21st International Conference on Computer Aided
Verification (CAV’09), Ahmed Bouajjani and Oded Maler (Eds.), Lecture Notes in Computer Science,
vol. 5643. Springer, 157–172.
Aaron R. Bradley and Zohar Manna. 2007. The Calculus of Computation. Springer.
Aaron R. Bradley, Zohar Manna, and Henny B. Sipma. 2006. What’s decidable about arrays? In Proceed-
ings of the 7th International Conference on Verification, Model Checking, and Abstract Interpretation
(VMCAI’06), E. Allen Emerson and Kedar S. Namjoshi (Eds.), Lecture Notes in Computer Science,
vol. 3855. Springer, 427–442.
Bruno Buchberger. 2006. Mathematical theory exploration. In Proceedings of the 3rd International Con-
ference on Automated Reasoning (IJCAR’66). Lecture Notes in Computer Science, vol. 4130. Springer,
1–2.
Bor-Yuh Evan Chang and K. Rustan M. Leino. 2005. Abstract Interpretation with alien expressions and
heap structures. In Proceedings of the 6th International Conference on Verification, Model Checking, and
Abstract Interpretation (VMCAI’05), Radhia Cousot (Ed.), Lecture Notes in Computer Science Series,
vol. 3385. Springer, 147–163.
Edmund M. Clarke, Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press.
Michael Colón, Sriram Sankaranarayanan, and Henny Sipma. 2003. Linear invariant generation using
non-linear constraint solving. In Proceedings of the 15th International Conference on Computer Aided
Verification(CAV’03), Jr. Warren A. Hunt and Fabio Somenzi (Eds.), Lecture Notes in Computer Science,
vol. 2725. Springer, 420–432.
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algo-
rithms (3rd ed.). MIT Press.
Patrick Cousot and Radhia Cousot. 1977. Abstract interpretation: A unified lattice model for static analysis
of programs by construction or approximation of fixpoints. In Proceedings of the 4th Annual ACM
Symposium on Principles of Programming Languages (POPL’77). 238–252.
Patrick Cousot and Nicolas Halbwachs. 1978. Automatic discovery of linear restraints among variables of a
program. In Proceedings of the 5th Annual ACM Symposium on Principles of Programming Languages
(POPL’78). 84–96.
Christoph Csallner, Nikolai Tillman, and Yannis Smaragdakis. 2008. DySy: Dynamic symbolic execution
for invariant inference. In Proceedings of the 30th International Conference on Software Engineering
(ICSE’08), Wilhelm Schäfer, Matthew B. Dwyer, and Volker Gruhn (Eds.). ACM, 281–290.
Guido de Caso, Diego Garbervetsky, and Daniel Gorı́n. 2009. Reducing the number of annotations in a
verification-oriented imperative language. In Proceedings of Automatic Program Verification.
Edsger W. Dijkstra. 1976. A Discipline of Programming. Prentice Hall.
Eiffel. 2006. Standard ECMA-367. Eiffel: Analysis, design and programming language. (2006).

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
34:50 C. A. Furia et al.

Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. 2001. Dynamically discovering
likely program invariants to support program evolution. IEEE Transactions of Software Engineering 27,
2 (2001), 99–123.
Robert W. Floyd. 1967. Assigning meanings to programs. In Mathematical Aspects of Computer Science J. T.
Schwartz (Ed.), Proceedings of Symposia in Applied Mathematics Series, vol. 19. American Mathematical
Society, 19–32.
Gordon Fraser and Andreas Zeller. 2011. Exploiting common object usage in test case generation. In Proceed-
ings of the IEEE 4th International Conference on Software Testing, Verification and Validation (ICST’11).
IEEE Computer Society, 80–89.
Carlo A. Furia, Dino Mandrioli, Angelo Morzenti, and Matteo Rossi. 2012. Modeling Time in Computing.
Monographs in Theoretical Computer Science. An CATCS Series. Springer.
Carlo A. Furia and Bertrand Meyer. 2010. Inferring loop invariants using postconditions. In Fields of Logic
and Computation: Essays Dedicated to Yuri Gurevich on the Occasion of His 70th Birthday, Andreas
Blass, Nachum Dershowitz, and Wolfgang Reisig (Eds.), Lecture Notes in Computer Science Series, vol.
6300. Springer, 277–300.
Michael R. Garey and David S. Johnson. 1979. Computers and Intractability: A Guide to the Theory of
NP-Completeness. Freeman.
Carlo Ghezzi, Andrea Mocci, and Mattia Monga. 2009. Synthesizing intensional behavior models by graph
transformation. In Proceedings of the 31st International Conference on Software Engineering (ICSE’09).
IEEE, 430–440.
David Gries. 1981. The Science of Programming. Springer-Verlag.
Neelam Gupta and Zachary V. Heidepriem. 2003. A new structural coverage criterion for dynamic detection
of program invariants. In 18th IEEE International Conference on Automated Software Engineering
(ASE’03). IEEE Computer Society, 49–59.
John Hatcliff, Gary T. Leavens, K. Rustan M. Leino, Peter Müller, and Matthew J. Parkinson. 2012. Behav-
ioral interface specification languages. ACM Comput. Surv. 44, 3 (2012), 16.
Thomas A. Henzinger, Thibaud Hottelier, Laura Kovács, and Andrei Voronkov. 2010. Invariant and type in-
ference for matrices. In Proceedings of the 11th International Conference on Verification, Model Checking,
and Abstract Interpretation (VMCAI’10). Lecture Notes in Computer Science. Springer.
C. A. R. Hoare. 1969. An axiomatic basis for computer programming. Commun. ACM 12, 10 (1969), 576–580.
C. A. R. Hoare. 1972. Proof of correctness of data representations. Acta Inf. 1 (1972), 271–281.
C. A. R. Hoare. 2003. The verifying compiler: A grand challenge for computing research. J. ACM 50, 1 (2003),
63–69.
Mikoláš Janota. 2007. Assertion-based loop invariant generation. In Proceedings of the 1st International
Workshop on Invariant Generation (WING’07).
Ranjit Jhala and Rupak Majumdar. 2009. Software model checking. Comput. Surveys 41, 4 (2009).
Michael Karr. 1976. Affine relationships among variables of a program. Acta Informatica 6 (1976), 133–151.
Hans Kellerer, Ulrich Pferschy, and David Pisinger. 2004. Knapsack problems. Springer.
Donald E. Knuth. 2011. The Art of Computer Programming (1st ed., Volumes 1–4A). Addison-Wesley. 1968–
1973.
Laura Kovács and Andrei Voronkov. 2009. Finding loop invariants for programs over arrays using a theorem
prover. In Proceedings of the 12th International Conference on Fundamental Approaches to Software
Engineering (FASE’09), Marsha Chechik and Martin Wirsing (Eds.), Lecture Notes in Computer Science
Series, vol. 5503. Springer, 470–485.
Daniel Kroening and Ofer Strichman. 2008. Decision Procedures: An Algorithmic Point of View. Monographs
in Theoretical Computer Sciences. An EATCS Series. Springer.
Shuvendu K. Lahiri, Shaz Qadeer, Juan P. Galeotti, Jan W. Voung, and Thomas Wies. 2009. Intra-module
Inference. In Proceedings of the 21st International Conference on Computer Aided Verification (CAV’09),
Ahmed Bouajjani and Oded Maler (Eds.), Lecture Notes in Computer Science Series, vol. 5643. Springer,
493–508.
Leslie Lamport. 1977. Proving the correctness of multiprocess programs. IEEE Transactions on Software
Engineering 3, 2 (1977), 125–143.
Leslie Lamport. 2009. Teaching concurrency. SIGACT News 40, 1 (2009), 58–62.
K. Rustan M. Leino. 2008. This is Boogie 2. (June 2008). (Manuscript KRML 178) http://research.microsoft.
com/en-us/projects/boogie/.
Francesco Logozzo. 2004. Automatic inference of class invariants. In Proceedings of the 5th International
Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI’04), Bernhard Steffen
and Giorgio Levi (Eds.), Lecture Notes in Computer Science Series, vol. 2937. Springer, 211–222.

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Loop Invariants: Analysis, Classification, and Examples 34:51

Kurt Mehlhorn and Peter Sanders. 2008. Algorithms and Data Structures: The Basic Toolbox. Springer.
Bertrand Meyer. 1980. A basis for the constructive approach to programming. In Proceedings of IFIP Congress
1980, Simon H. Lavington (Ed.). 293–298.
Bertrand Meyer. 1997. Object-oriented software construction (2nd ed.). Prentice Hall.
Carroll Morgan. 1994. Programming from Specifications (2nd ed.). Prentice Hall.
ThanhVu Nguyen, Deepak Kapur, Westley Weimer, and Stephanie Forrest. 2012. Using dynamic analysis
to discover polynomial and array invariants. In Proceedings of the 34th International Conference on
Software Engineering (ICSE’12). IEEE, 683–693.
Susan Owicki and David Gries. 1976. An axiomatic proof technique for parallel programs. Acta Informatica
6 (1976), 319–340.
Christos H. Papadimitriou. 1993. Computational Complexity. Addison-Wesley.
Corina S. Păsăreanu and Willem Visser. 2004. Verification of Java programs using symbolic execution
and invariant generation. In Proceedings of the 11th International SPIN Workshop on Model Checking
Software. Lecture Notes in Computer Science Series, vol. 2989. Springer, 164–181.
Jeff H. Perkings and Michael D. Ernst. 2004. Efficient incremental algorithms for dynamic detection of
likely invariants. In Proceedings of the 12th ACM SIGSOFT International Symposium on Foundations
of Software Engineering (SIGSOFT’04/FSE-12), Richard N. Taylor and Matthew B. Dwyer (Eds.). ACM,
23–32.
André Platzer. 2010. Differential-algebraic dynamic logic for differential-algebraic programs. J. Log. Comput.
20, 1 (2010), 309–352.
Nadia Polikarpova, Ilinca Ciupa, and Bertrand Meyer. 2009. A comparative study of programmer-written
and automatically inferred contracts. In Proceedings of the ACM/SIGSOFT International Symposium
on Software Testing and Analysis (ISSTA’09). 93–104.
Alan Robinson and Andrei Voronkov (Eds.). 2001. Handbook of Automated Reasoning. Elsevier.
Enric Rodrı́guez-Carbonell and Deepak Kapur. 2007. Generating all polynomial invariants in simple loops.
Journal of Symbolic Computation 42, 4 (2007), 443–476.
Sriram Sankaranarayanan, Henny Sipma, and Zohar Manna. 2004. Non-linear loop invariant generation
using Gröbner bases. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of
Programming Languages (POPL’04), Neil D. Jones and Xavier Leroy (Eds.). ACM, 318–329.
Michael I. Shamos. 1978. Computational geometry. Ph.D.thesis Yale University. http://goo.gl/XiXNl.
J. Michael Spivey. 1992. Z Notation – a reference manual (2nd ed.). Prentice Hall International Series in
Computer Science. Prentice Hall.
Nikolai Tillmann, Feng Chen, and Wolfram Schulte. 2006. Discovering likely method specifications. In
Proceedings of the 8th International Conference on Formal Engineering Methods (ICFEM’06). Lecture
Notes in Computer Science Series, vol. 4260. 717–736.
Godfried T. Toussaint. 1983. Solving geometric problems with the rotating calipers. In Proceedings of IEEE
MELECON.
Yi Wei, Carlo A. Furia, Nikolay Kazmin, and Bertrand Meyer. 2011a. Inferring Better Contracts. In Proceed-
ings of the 33rd International Conference on Software Engineering (ICSE’11), Richard N. Taylor, Harald
Gall, and Nenad Medvidović (Eds.). ACM, 191–200.
Yi Wei, Hannes Roth, Carlo A. Furia, Yu Pei, Alexander Horton, Michael Steindorfer, Martin Nordio, and
Bertrand Meyer. 2011b. Stateful testing: Finding more errors in code and contracts. In Proceedings
of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE’11), Perry
Alexander, Corina Pasareanu, and John Hosking (Eds.). ACM, 440–443.

Received November 2012; revised June 2013; accepted July 2013

ACM Computing Surveys, Vol. 46, No. 3, Article 34, Publication date: January 2014.
Copyright of ACM Computing Surveys is the property of Association for Computing
Machinery and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.