CMS Books in Mathematics CMS Books in Mathematics

Heinz H. Bauschke · Patrick L. Combettes

Bauschke · Combettes
Convex Analysis and Monotone Operator Theory in Hilbert Spaces

This book presents a largely self-contained account of the main results of convex analysis,
monotone operator theory, and the theory of nonexpansive operators in the context
of Hilbert spaces. Unlike existing literature, the novelty of this book, and indeed its Canadian
central theme, is the tight interplay among the key notions of convexity, monotonicity,
Mathematical Society
and nonexpansiveness. The presentation is accessible to a broad audience and attempts
to reach out in particular to the applied sciences and engineering communities, where Heinz H. Bauschke Société mathématique
these tools have become indispensable. Graduate students and researchers in pure and
applied mathematics will benefit from this book. It is also directed to researchers in
engineering, decision sciences, economics, and inverse problems, and can serve as a
Patrick L. Combettes du Canada

reference book.

1 Convex Analysis
and Monotone

Convex Analysis and Monotone Operator Theory in Hilbert Spaces

Operator Theory
in Hilbert Spaces
Mathematics
isbn 978-1-4419-9466-0

9 78 1 44 1 994660
Foreword

This self-contained book offers a modern unifying presentation of three ba-

sic areas of nonlinear analysis, namely convex analysis, monotone operator
theory, and the fixed point theory of nonexpansive mappings.
This turns out to be a judicious choice. Showing the rich connections
and interplay between these topics gives a strong coherence to the book.
Moreover, these particular topics are at the core of modern optimization and
its applications.
Choosing to work in Hilbert spaces offers a wide range of applications,
while keeping the mathematics accessible to a large audience. Each topic is
developed in a self-contained fashion, and the presentation often draws on
recent advances.
The organization of the book makes it accessible to a large audience. Each
chapter is illustrated by several exercises, which makes the monograph an
excellent textbook. In addition, it offers deep insights into algorithmic aspects
of optimization, especially splitting algorithms, which are important in theory
and applications.
Let us point out the high quality of the writing and presentation. The au-
thors combine an uncompromising demand for rigorous mathematical state-
ments and a deep concern for applications, which makes this book remarkably
accomplished.

Montpellier (France), October 2010 Hédy Attouch

vii
Preface

Three important areas of nonlinear analysis emerged in the early 1960s: con-
vex analysis, monotone operator theory, and the theory of nonexpansive map-
pings. Over the past four decades, these areas have reached a high level of
maturity, and an increasing number of connections have been identified be-
tween them. At the same time, they have found applications in a wide ar-
ray of disciplines, including mechanics, economics, partial differential equa-
tions, information theory, approximation theory, signal and image process-
ing, game theory, optimal transport theory, probability and statistics, and
machine learning.
The purpose of this book is to present a largely self-contained account
of the main results of convex analysis, monotone operator theory, and the
theory of nonexpansive operators in the context of Hilbert spaces. Authori-
tative monographs are already available on each of these topics individually.
A novelty of this book, and indeed, its central theme, is the tight interplay
among the key notions of convexity, monotonicity, and nonexpansiveness. We
aim at making the presentation accessible to a broad audience, and to reach
out in particular to the applied sciences and engineering communities, where
these tools have become indispensable. We chose to cast our exposition in the
Hilbert space setting. This allows us to cover many applications of interest
to practitioners in infinite-dimensional spaces and yet to avoid the technical
difficulties pertaining to general Banach space theory that would exclude a
large portion of our intended audience. We have also made an attempt to
draw on recent developments and modern tools to simplify the proofs of key
results, exploiting for instance heavily the concept of a Fitzpatrick function
in our exposition of monotone operators, the notion of Fejér monotonicity to
unify the convergence proofs of several algorithms, and that of a proximity
operator throughout the second half of the book.
The book in organized in 29 chapters. Chapters 1 and 2 provide back-
ground material. Chapters 3 to 7 cover set convexity and nonexpansive op-
erators. Various aspects of the theory of convex functions are discussed in
Chapters 8 to 19. Chapters 20 to 25 are dedicated to monotone operator the-

ix
x Preface

ory. In addition to these basic building blocks, we also address certain themes
from different angles in several places. Thus, optimization theory is discussed
in Chapters 11, 19, 26, and 27. Best approximation problems are discussed
in Chapters 3, 19, 27, 28, and 29. Algorithms are also present in various
parts of the book: fixed point and convex feasibility algorithms in Chap-
ter 5, proximal-point algorithms in Chapter 23, monotone operator splitting
algorithms in Chapter 25, optimization algorithms in Chapter 27, and best
approximation algorithms in Chapters 27 and 29. More than 400 exercises
are distributed throughout the book, at the end of each chapter.
Preliminary drafts of this book have been used in courses in our institu-
tions and we have benefited from the input of postdoctoral fellows and many
students. To all of them, many thanks. In particular, HHB thanks Liangjin
Yao for his helpful comments. We are grateful to Hédy Attouch, Jon Borwein,
Stephen Simons, Jon Vanderwerff, Shawn Wang, and Isao Yamada for helpful
discussions and pertinent comments. PLC also thanks Oscar Wesler. Finally,
we thank the Natural Sciences and Engineering Research Council of Canada,
the Canada Research Chair Program, and France’s Agence Nationale de la
Recherche for their support.

Kelowna (Canada) Heinz H. Bauschke

Paris (France) Patrick L. Combettes
October 2010
Contents

1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Sets in Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 The Extended Real Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.7 Topological Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.8 Two–Point Compactification of the Real Line . . . . . . . . . . . . . . 9
1.9 Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.10 Lower Semicontinuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.11 Sequential Topological Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.12 Metric Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1 Notation and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Basic Identities and Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3 Linear Operators and Functionals . . . . . . . . . . . . . . . . . . . . . . . . 31
2.4 Strong and Weak Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.5 Weak Convergence of Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.6 Differentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3 Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1 Definition and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Best Approximation Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Topological Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4 Separation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

xi
xii Contents

4 Convexity and Nonexpansiveness . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.1 Nonexpansive Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2 Projectors onto Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.3 Fixed Points of Nonexpansive Operators . . . . . . . . . . . . . . . . . . . 62
4.4 Averaged Nonexpansive Operators . . . . . . . . . . . . . . . . . . . . . . . . 67
4.5 Common Fixed Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5 Fejér Monotonicity and Fixed Point Iterations . . . . . . . . . . . . 75

5.1 Fejér Monotone Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2 Krasnosel’skiı̆–Mann Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.3 Iterating Compositions of Averaged Operators . . . . . . . . . . . . . 82
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6 Convex Cones and Generalized Interiors . . . . . . . . . . . . . . . . . . 87

6.1 Convex Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
6.2 Generalized Interiors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.3 Polar and Dual Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.4 Tangent and Normal Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.5 Recession and Barrier Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

7 Support Functions and Polar Sets . . . . . . . . . . . . . . . . . . . . . . . . 107

7.1 Support Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.2 Support Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
7.3 Polar Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

8 Convex Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

8.1 Definition and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
8.2 Convexity–Preserving Operations . . . . . . . . . . . . . . . . . . . . . . . . . 116
8.3 Topological Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

9 Lower Semicontinuous Convex Functions . . . . . . . . . . . . . . . . . 129

9.1 Lower Semicontinuous Convex Functions . . . . . . . . . . . . . . . . . . 129
9.2 Proper Lower Semicontinuous Convex Functions . . . . . . . . . . . . 132
9.3 Affine Minorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
9.4 Construction of Functions in Γ0 (H) . . . . . . . . . . . . . . . . . . . . . . . 136
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

10 Convex Functions: Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

10.1 Between Linearity and Convexity . . . . . . . . . . . . . . . . . . . . . . . . . 143
10.2 Uniform and Strong Convexity . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
10.3 Quasiconvexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Contents xiii

11 Convex Variational Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

11.1 Infima and Suprema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
11.2 Minimizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
11.3 Uniqueness of Minimizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
11.4 Existence of Minimizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
11.5 Minimizing Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

12 Infimal Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

12.1 Definition and Basic Facts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
12.2 Infimal Convolution of Convex Functions . . . . . . . . . . . . . . . . . . 170
12.3 Pasch–Hausdorff Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
12.4 Moreau Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
12.5 Infimal Postcomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

13 Conjugation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
13.1 Definition and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
13.2 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
13.3 The Fenchel–Moreau Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

14 Further Conjugation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

14.1 Moreau’s Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
14.2 Proximal Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
14.3 Positively Homogeneous Functions . . . . . . . . . . . . . . . . . . . . . . . . 201
14.4 Coercivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
14.5 The Conjugate of the Difference . . . . . . . . . . . . . . . . . . . . . . . . . . 204
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

15 Fenchel–Rockafellar Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

15.1 The Attouch–Brézis Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
15.2 Fenchel Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
15.3 Fenchel–Rockafellar Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
15.4 A Conjugation Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
15.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

16 Subdifferentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
16.1 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
16.2 Convex Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
16.3 Lower Semicontinuous Convex Functions . . . . . . . . . . . . . . . . . . 229
16.4 Subdifferential Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
xiv Contents

17 Differentiability of Convex Functions . . . . . . . . . . . . . . . . . . . . . 241

17.1 Directional Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
17.2 Characterizations of Convexity . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
17.3 Characterizations of Strict Convexity . . . . . . . . . . . . . . . . . . . . . 246
17.4 Directional Derivatives and Subgradients . . . . . . . . . . . . . . . . . . 247
17.5 Gâteaux and Fréchet Differentiability . . . . . . . . . . . . . . . . . . . . . 251
17.6 Differentiability and Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

18 Further Differentiability Results . . . . . . . . . . . . . . . . . . . . . . . . . . 261

18.1 The Ekeland–Lebourg Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 261
18.2 The Subdifferential of a Maximum . . . . . . . . . . . . . . . . . . . . . . . . 264
18.3 Differentiability of Infimal Convolutions . . . . . . . . . . . . . . . . . . . 266
18.4 Differentiability and Strict Convexity . . . . . . . . . . . . . . . . . . . . . 267
18.5 Stronger Notions of Differentiability . . . . . . . . . . . . . . . . . . . . . . 268
18.6 Differentiability of the Distance to a Set . . . . . . . . . . . . . . . . . . . 271
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

19 Duality in Convex Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 275

19.1 Primal Solutions via Dual Solutions . . . . . . . . . . . . . . . . . . . . . . . 275
19.2 Parametric Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
19.3 Minimization under Equality Constraints . . . . . . . . . . . . . . . . . . 283
19.4 Minimization under Inequality Constraints . . . . . . . . . . . . . . . . . 285
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

20 Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

20.1 Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
20.2 Maximally Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . . . . 297
20.3 Bivariate Functions and Maximal Monotonicity . . . . . . . . . . . . 302
20.4 The Fitzpatrick Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

21 Finer Properties of Monotone Operators . . . . . . . . . . . . . . . . . 311

21.1 Minty’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
21.2 The Debrunner–Flor Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
21.3 Domain and Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
21.4 Local Boundedness and Surjectivity . . . . . . . . . . . . . . . . . . . . . . . 318
21.5 Kenderov’s Theorem and Fréchet Differentiability . . . . . . . . . . 320
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

22 Stronger Notions of Monotonicity . . . . . . . . . . . . . . . . . . . . . . . . 323

22.1 Para, Strict, Uniform, and Strong Monotonicity . . . . . . . . . . . . 323
22.2 Cyclic Monotonicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
22.3 Rockafellar’s Cyclic Monotonicity Theorem . . . . . . . . . . . . . . . . 327
22.4 Monotone Operators on R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
Contents xv

23 Resolvents of Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . 333

23.1 Definition and Basic Identities . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
23.2 Monotonicity and Firm Nonexpansiveness . . . . . . . . . . . . . . . . . 335
23.3 Resolvent Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
23.4 Zeros of Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
23.5 Asymptotic Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

24 Sums of Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

24.1 Maximal Monotonicity of a Sum . . . . . . . . . . . . . . . . . . . . . . . . . . 351
24.2 3∗ Monotone Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
24.3 The Brézis–Haraux Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
24.4 Parallel Sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

25 Zeros of Sums of Monotone Operators . . . . . . . . . . . . . . . . . . . . 363

25.1 Characterizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
25.2 Douglas–Rachford Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
25.3 Forward–Backward Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
25.4 Tseng’s Splitting Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
25.5 Variational Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378

26 Fermat’s Rule in Convex Optimization . . . . . . . . . . . . . . . . . . . 381

26.1 General Characterizations of Minimizers . . . . . . . . . . . . . . . . . . . 381
26.2 Abstract Constrained Minimization Problems . . . . . . . . . . . . . . 383
26.3 Affine Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
26.4 Cone Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
26.5 Convex Inequality Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
26.6 Regularization of Minimization Problems . . . . . . . . . . . . . . . . . . 393
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

27 Proximal Minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

27.1 The Proximal-Point Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
27.2 Douglas–Rachford Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
27.3 Forward–Backward Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
27.4 Tseng’s Splitting Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
27.5 A Primal–Dual Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

28 Projection Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

28.1 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
28.2 Projections onto Affine Subspaces . . . . . . . . . . . . . . . . . . . . . . . . 417
28.3 Projections onto Special Polyhedra . . . . . . . . . . . . . . . . . . . . . . . 419
28.4 Projections Involving Convex Cones . . . . . . . . . . . . . . . . . . . . . . 425
28.5 Projections onto Epigraphs and Lower Level Sets . . . . . . . . . . . 427
xvi Contents

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429

29 Best Approximation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 431

29.1 Dykstra’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
29.2 Haugazeau’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440

Bibliographical Pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

Symbols and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
Chapter 5
Fejér Monotonicity and Fixed Point
Iterations

A sequence is Fejér monotone with respect to a set C if each point in the

sequence is not strictly farther from any point in C than its predecessor. Such
sequences possess very attractive properties that greatly simplify the analy-
sis of their asymptotic behavior. In this chapter, we provide the basic theory
for Fejér monotone sequences and apply it to obtain in a systematic fash-
ion convergence results for various classical iterations involving nonexpansive
operators.

5.1 Fejér Monotone Sequences

The following notion is central in the study of various iterative methods, in

particular in connection with the construction of fixed points of nonexpansive
operators.

Definition 5.1 Let C be a nonempty subset of H and let (xn )n∈N be a

sequence in H. Then (xn )n∈N is Fejér monotone with respect to C if

(∀x ∈ C)(∀n ∈ N) kxn+1 − xk ≤ kxn − xk. (5.1)

Example 5.2 Let (xn )n∈N be a bounded sequence in R that is increasing

(respectively decreasing). Then (xn )n∈N is Fejér monotone with respect to
[sup{xn }n∈N , +∞[ (respectively ]−∞, inf{xn }n∈N ]).

Example 5.3 Let D be a nonempty subset of H, let T : D → D be a

quasinonexpansive—in particular, nonexpansive—operator such that Fix T 6=
∅, and let x0 ∈ D. Set (∀n ∈ N) xn+1 = T xn . Then (xn )n∈N is Fejér mono-
tone with respect to Fix T .

We start with some basic properties.

75
76 5 Fejér Monotonicity and Fixed Point Iterations

Proposition 5.4 Let (xn )n∈N be a sequence in H and let C be a nonempty

subset of H. Suppose that (xn )n∈N is Fejér monotone with respect to C. Then
the following hold:
(i) (xn )n∈N is bounded.
(ii) For every x ∈ C, (kxn − xk)n∈N converges.
(iii) (dC (xn ))n∈N is decreasing and converges.
Proof. (i): Let x ∈ C. Then (5.1) implies that (xn )n∈N lies in B(x; kx0 − xk).
(ii): Clear from (5.1).
(iii): Taking the infimum in (5.1) over x ∈ C yields (∀n ∈ N) dC (xn+1 ) ≤
dC (xn ). ⊓
⊔
The next result concerns weak convergence.
Theorem 5.5 Let (xn )n∈N be a sequence in H and let C be a nonempty sub-
set of H. Suppose that (xn )n∈N is Fejér monotone with respect to C and that
every weak sequential cluster point of (xn )n∈N belongs to C. Then (xn )n∈N
converges weakly to a point in C.
Proof. The result follows from Proposition 5.4(ii) and Lemma 2.39. ⊓
⊔
Example 5.6 Suppose that H is infinite-dimensional and let (xn )n∈N be an
orthonormal sequence in H. Then (xn )n∈N is Fejér monotone with respect to
{0}. As seen in Example 2.25, xn ⇀ 0 but xn 6→ 0.
While a Fejér monotone sequence with respect to a closed convex set C
may not converge strongly, its “shadow” on C always does.
Proposition 5.7 Let (xn )n∈N be a sequence in H and let C be a nonempty
closed convex subset of H. Suppose that (xn )n∈N is Fejér monotone with re-
spect to C. Then the shadow sequence (PC xn )n∈N converges strongly to a
point in C.
Proof. It follows from (5.1) and (3.6) that, for every m and n in N,

kPC xn − PC xn+m k2 = kPC xn − xn+m k2 + kxn+m − PC xn+m k2

+ 2 hPC xn − xn+m | xn+m − PC xn+m i
≤ kPC xn − xn k2 + d2C (xn+m )
+ 2 hPC xn − PC xn+m | xn+m − PC xn+m i
+ 2 hPC xn+m − xn+m | xn+m − PC xn+m i
≤ d2C (xn ) − d2C (xn+m ). (5.2)

Consequently, since (dC (xn ))n∈N was seen in Proposition 5.4(iii) to converge,
(PC xn )n∈N is a Cauchy sequence in the complete set C. ⊓
⊔
Corollary 5.8 Let (xn )n∈N be a sequence in H, let C be a nonempty closed
convex subset of H, and let x ∈ C. Suppose that (xn )n∈N is Fejér monotone
with respect to C and that xn ⇀ x. Then PC xn → x.
5.1 Fejér Monotone Sequences 77

Proof. By Proposition 5.7, (PC xn )n∈N converges strongly to some point y ∈

C. Hence, since x − PC xn → x − y and xn − PC xn ⇀ x − y, it follows
from Theorem 3.14 and Lemma 2.41(iii) that 0 ≥ hx − PC xn | xn − PC xn i →
kx − yk2 . Thus, x = y. ⊓
⊔
For sequences that are Fejér monotone with respect to closed affine sub-
spaces, Proposition 5.7 can be strengthened.

Proposition 5.9 Let (xn )n∈N be a sequence in H and let C be a closed affine
subspace of H. Suppose that (xn )n∈N is Fejér monotone with respect to C.
Then the following hold:
(i) (∀n ∈ N) PC xn = PC x0 .
(ii) Suppose that every weak sequential cluster point of (xn )n∈N belongs to
C. Then xn ⇀ PC x0 .

Proof. (i): Fix n ∈ N, α ∈ R, and set yα = αPC x0 + (1 − α)PC xn . Since C

is an affine subspace, yα ∈ C, and it therefore follows from Corollary 3.20(i)
and (5.1) that

α2 kPC xn − PC x0 k2 = kPC xn − yα k2
≤ kxn − PC xn k2 + kPC xn − yα k2
= kxn − yα k2
≤ kx0 − yα k2
= kx0 − PC x0 k2 + kPC x0 − yα k2
= d2C (x0 ) + (1 − α)2 kPC xn − PC x0 k2 . (5.3)

Consequently, (2α − 1)kPC xn − PC x0 k2 ≤ d2C (x0 ) and, letting α → +∞, we

conclude that PC xn = PC x0 .
(ii): Combine Theorem 5.5, Corollary 5.8, and (i). ⊓
⊔
We now turn our attention to strong convergence properties.

Proposition 5.10 Let (xn )n∈N be a sequence in H and let C be a subset of

H such that int C 6= ∅. Suppose that (xn )n∈N is Fejér monotone with respect
to C. Then (xn )n∈N converges strongly to a point in H.

Proof. Take x ∈ int C and ρ ∈ R++ such that B(x; ρ) ⊂ C. Define a sequence
(zn )n∈N in B(x; ρ) by

x, if xn+1 = xn ;
(∀n ∈ N) zn = xn+1 − xn (5.4)
x − ρ , otherwise.
kxn+1 − xn k

Then (5.1) yields (∀n ∈ N) kxn+1 − zn k2 ≤ kxn − zn k2 and, after expanding,

we obtain
78 5 Fejér Monotonicity and Fixed Point Iterations

(∀n ∈ N) kxn+1 − xk2 ≤ kxn − xk2 − 2ρkxn+1 − xn k. (5.5)

Thus, n∈N kxn+1 −xn k ≤ kx0 −xk2 /(2ρ) and (xn )n∈N is therefore a Cauchy
P
sequence. ⊓
⊔

Theorem 5.11 Let (xn )n∈N be a sequence in H and let C be a nonempty

closed convex subset of H. Suppose that (xn )n∈N is Fejér monotone with re-
spect to C. Then the following are equivalent:
(i) (xn )n∈N converges strongly to a point in C.
(ii) (xn )n∈N possesses a strong sequential cluster point in C.
(iii) lim dC (xn ) = 0.

Proof. (i)⇒(ii): Clear.

(ii)⇒(iii): Suppose that xkn → x ∈ C. Then dC (xkn ) ≤ kxkn − xk → 0.
(iii)⇒(i): Proposition 5.4(iii) implies that dC (xn ) → 0. Hence, xn −
PC xn → 0 and (i) follows from Proposition 5.7. ⊓
⊔
We conclude this section with a linear convergence result.

Theorem 5.12 Let (xn )n∈N be a sequence in H and let C be a nonempty

closed convex subset of H. Suppose that (xn )n∈N is Fejér monotone with re-
spect to C and that for some κ ∈ [0, 1[,

(∀n ∈ N) dC (xn+1 ) ≤ κdC (xn ). (5.6)

Then (xn )n∈N converges linearly to a point x ∈ C; more precisely,

(∀n ∈ N) kxn − xk ≤ 2κn dC (x0 ). (5.7)

Proof. Theorem 5.11 and (5.6) imply that (xn )n∈N converges strongly to some
point x ∈ C. On the other hand, (5.1) yields

(∀n ∈ N)(∀m ∈ N) kxn − xn+m k ≤ kxn − PC xn k + kxn+m − PC xn k

≤ 2dC (xn ). (5.8)

Letting m → +∞ in (5.8), we conclude that kxn − xk ≤ 2dC (xn ). ⊓

⊔

5.2 Krasnosel’skiı̆–Mann Iteration

Given a nonexpansive operator T , the sequence generated by the Banach–

Picard iteration xn+1 = T xn of (1.66) may fail to produce a fixed point of
T . A simple illustration of this situation is T = −Id and x0 6= 0. In this case,
however, it is clear that the asymptotic regularity property xn − T xn → 0
does not hold. As we shall now see, this property is critical.
5.2 Krasnosel’skiı̆–Mann Iteration 79

Theorem 5.13 Let D be a nonempty closed convex subset of H, let T : D →

D be a nonexpansive operator such that Fix T 6= ∅, and let x0 ∈ D. Set

(∀n ∈ N) xn+1 = T xn (5.9)

and suppose that xn − T xn → 0. Then the following hold:

(i) (xn )n∈N converges weakly to a point in Fix T .
(ii) Suppose that D = −D and that T is odd: (∀x ∈ D) T (−x) = −T x.
Then (xn )n∈N converges strongly to a point in Fix T .

Proof. From Example 5.3, (xn )n∈N is Fejér monotone with respect to Fix T .
(i): Let x be a weak sequential cluster point of (xn )n∈N , say xkn ⇀ x.
Since T xkn − xkn → 0, Corollary 4.18 asserts that x ∈ Fix T . Appealing to
Theorem 5.5, the assertion is proved.
(ii): Since D = −D is convex, 0 ∈ D and, since T is odd, 0 ∈ Fix T .
Therefore, by Fejér monotonicity, (∀n ∈ N) kxn+1 k ≤ kxn k. Thus, there
exists ℓ ∈ R+ such that kxn k ↓ ℓ. Now let m ∈ N. Then, for every n ∈ N,

kxn+1+m + xn+1 k = kT xn+m − T (−xn )k ≤ kxn+m + xn k, (5.10)

and, by the parallelogram identity,

kxn+m + xn k2 = 2 kxn+m k2 + kxm k2 − kxn+m − xn k2 .

(5.11)

However, since T xn − xn → 0, we have limn kxn+m − xn k = 0. Therefore,

since kxn k ↓ ℓ, (5.10) and (5.11) yield kxn+m + xn k ↓ 2ℓ as n → +∞. In turn,
we derive from (5.11) that kxn+m − xn k2 ≤ 2(kxn+m k2 + kxm k2 ) − 4ℓ2 → 0
as m, n → +∞. Thus, (xn )n∈N is a Cauchy sequence and xn → x for some
x ∈ D. Since xn+1 → x and xn+1 = T xn → T x, we have x ∈ Fix T . ⊓
⊔
We now turn our attention to an alternative iterative method, known as
the Krasnosel’skiı̆–Mann algorithm.

Theorem 5.14 (Krasnosel’skiı̆–Mann algorithm) Let D be a nonempty

closed convex subset of H, let T : D → D be a nonexpansivePoperator such that
Fix T 6= ∅, let (λn )n∈N be a sequence in [0, 1] such that n∈N λn (1 − λn ) =
+∞, and let x0 ∈ D. Set


(∀n ∈ N) xn+1 = xn + λn T xn − xn . (5.12)

Then the following hold:

(i) (xn )n∈N is Fejér monotone with respect to Fix T .
(ii) (T xn − xn )n∈N converges strongly to 0.
(iii) (xn )n∈N converges weakly to a point in Fix T .

Proof. Since x0 ∈ D and D is convex, (5.12) produces a well-defined sequence

in D.
80 5 Fejér Monotonicity and Fixed Point Iterations

(i): It follows from Corollary 2.14 and the nonexpansiveness of T that, for
every y ∈ Fix T and n ∈ N,

kxn+1 − yk2 = k(1 − λn )(xn − y) + λn (T xn − y)k2

= (1 − λn )kxn − yk2 + λn kT xn − T yk2
− λn (1 − λn )kT xn − xn k2
≤ kxn − yk2 − λn (1 − λn )kT xn − xn k2 . (5.13)

Hence, (xn )n∈N is Fejér monotone P

with respect to Fix T .
We derive from (5.13) that n∈N λn (1 − λn )kT xn − xn k2 ≤ kx0 − yk2 .
(ii):P
Since n∈N λn (1 − λn ) = +∞, we have lim kT xn − xn k = 0. However, for
every n ∈ N,

kT xn+1 − xn+1 k = kT xn+1 − T xn + (1 − λn )(T xn − xn )k

≤ kxn+1 − xn k + (1 − λn )kT xn − xn k
= kT xn − xn k. (5.14)

Consequently, (kT xn − xn k)n∈N converges and we must have T xn − xn → 0.

(iii): Let x be a weak sequential cluster point of (xn )n∈N , say xkn ⇀ x.
Then it follows from Corollary 4.18 that x ∈ Fix T . In view of Theorem 5.5,
the proof is complete. ⊓
⊔

Proposition 5.15 Let α ∈ ]0, 1[, let T : H → H be an α-averaged oper-

P such that Fix T 6= ∅, let (λn )n∈N be a sequence in [0, 1/α] such that
ator
n∈N λn (1 − αλn ) = +∞, and let x0 ∈ H. Set


(∀n ∈ N) xn+1 = xn + λn T xn − xn . (5.15)

Then the following hold:

(i) (xn )n∈N is Fejér monotone with respect to Fix T .
(ii) (T xn − xn )n∈N converges strongly to 0.
(iii) (xn )n∈N converges weakly to a point in Fix T .

Proof. Set R = (1 − 1/α)Id + (1/α)T and (∀n ∈ N) µn = αλn . Then Fix R =

Fix T and R is nonexpansive by Proposition 4.25.

In addition, we rewrite
(5.15)
P as (∀n ∈ N) xn+1 = xn + µ n Rxn − xn . Since (µn )n∈N lies in [0, 1]
and n∈N µn (1 − µn ) = +∞, the results follow from Theorem 5.14. ⊓
⊔

Corollary 5.16 Let T : H → H be a firmly nonexpansivePoperator such that

Fix T 6= ∅, let (λn )n∈N be a sequence in [0, 2] such that n∈N λn
(2 − λn ) =
+∞, and let x0 ∈ H. Set (∀n ∈ N) xn+1 = xn + λn T xn − xn . Then the
following hold:
(i) (xn )n∈N is Fejér monotone with respect to Fix T .
(ii) (T xn − xn )n∈N converges strongly to 0.
5.2 Krasnosel’skiı̆–Mann Iteration 81

(iii) (xn )n∈N converges weakly to a point in Fix T .

Proof. In view of Remark 4.24(iii), apply Proposition 5.15 with α = 1/2. ⊓

⊔

Example 5.17 Let T : H → H be a firmly nonexpansive operator such that

Fix T 6= ∅, let x0 ∈ H, and set (∀n ∈ N) xn+1 = T xn . Then (xn )n∈N
converges weakly to a point in Fix T .

The following type of iterative method involves a mix of compositions and

convex combinations of nonexpansive operators.

Corollary 5.18 Let (Ti )i∈I T be a finite family of nonexpansive operators

from H to H such that i∈I Fix Ti 6= ∅, and let (αi )i∈I be real num-
bers in ]0, 1[ such that, for every i ∈ I, Ti is αi -averaged. Let p be a
strictly positive integer, for every k ∈ {1, . . . , p}, let mk be a strictly
positive integer and ωk be a strictly positive real number, and suppose
that
Pp i : (k, l)  k ∈ {1, . . . , p}, l ∈ {1, . . . , mk } → I is surjective and that
k=1 ωk = 1. For every k ∈ {1, . . . , p}, set Ik = {i(k, 1), . . . , i(k, mk )}, and
set
mk
α = max ρk , where (∀k ∈ {1, . . . , p}) ρk = , (5.16)
1≤k≤p 1
mk − 1 +
max αi
i∈Ik
P
and let (λn )n∈N be a sequence in [0, 1/α] such that n∈N λn (1 − αλn ) = +∞.
Furthermore, let x0 ∈ H and set
p
!
X
(∀n ∈ N) xn+1 = xn + λn ωk Ti(k,1) · · · Ti(k,mk ) xn − xn . (5.17)
k=1
T
Then (xn )n∈N converges weakly to a point in i∈I Fix Ti .
Pp
Proof. Set T = k=1 ωk Rk , where (∀k ∈ {1, . . . , p}) Rk = Ti(k,1) · · · Ti(k,mk ) .
Then (5.17) reduces to (5.15) and, in view of Proposition
T 5.15, it suffices
to show that T is α-averaged and that Fix T = i∈I Fix Ti . For every k ∈
{1, . . . , p}, it follows from Proposition 4.32
T and (5.16) that Rk is ρk -averaged
and, from Corollary 4.37 that Fix Rk = i∈Ik Fix Ti . In turn, we derive from
Proposition 4.30 Tp and (5.16) that
Tp T T is α-averaged and,
T from Proposition 4.34,
that Fix T = k=1 Fix Rk = k=1 i∈Ik Fix Ti = i∈I Fix Ti . ⊓
⊔

Remark 5.19 It follows from Remark 4.24(iii) that Corollary 5.18 is appli-
cable to firmly nonexpansive operators and, a fortiori, to projection operators
by Proposition 4.8.

Corollary 5.18 provides an algorithm to solve a convex feasibility problem,

i.e., to find a point in the intersection of a family of closed convex sets. Here
are two more examples.
82 5 Fejér Monotonicity and Fixed Point Iterations

Example 5.20 (string-averaged relaxed projections) T Let (Ci )i∈I be

a finite family of closed convex sets such that C = i∈I Ci 6= ∅. For
every i ∈ I, let βi ∈ ]0, 2[ and set Ti = (1 − βi )Id + βi PCi . Let p be
a strictly positive integer; for every k ∈ {1, . . . , p}, let mk be a strictly
positive integer and ωk be a strictly positive real number, and suppose
that
Pp i : (k, l)  k ∈ {1, . . . , p}, l ∈ {1, . . . , mk } → I is surjective and that
k=1 ωk = 1. Furthermore, let x0 ∈ H and set

p
X
(∀n ∈ N) xn+1 = ωk Ti(k,1) · · · Ti(k,mk ) xn . (5.18)
k=1

Then (xn )n∈N converges weakly to a point in C.

Proof. For every i ∈ I, set αi = βi /2 ∈ ]0, 1[. Since, for every i ∈ I, Proposi-
tion 4.8 asserts that PCi is firmly nonexpansive, Corollary 4.29 implies that
Ti is αi -averaged. Borrowing notation from Corollary 5.18, we note that for
every k ∈ {1, . . . , p}, maxi∈Ik αi ∈ ]0, 1[, which implies that ρk ∈ ]0, 1[ and
thus that α ∈ ]0, 1[. Altogether, the result follows from Corollary 5.18 with
λn ≡ 1. ⊓
⊔

Example 5.21 (parallel projection algorithm)T Let (Ci )i∈I be a finite

family of closed convex subsets of HPsuch that C = i∈I Ci 6= ∅, let (λn )n∈N
be a sequence in [0, 2] such that λn (2 − λn ) = +∞, let (ωi )i∈I be
n∈NP
strictly positive real numbers such that i∈I ωi = 1, and let x0 ∈ H. Set
X 
(∀n ∈ N) xn+1 = xn + λn ωi Pi xn − xn , (5.19)
i∈I

where, for every i ∈ I, Pi denotes the projector onto Ci . Then (xn )n∈N
converges weakly to a point in C.
P
Proof. This is an application of Corollary 5.16(iii) with T = i∈I ωi Pi . In-
deed, since the operators (Pi )i∈I are firmly nonexpansive by Proposition 4.8,
their convex combination T is also firmly nonexpansive
T by Example
T 4.31.
Moreover, Proposition 4.34 asserts that Fix T = i∈I Fix Pi = i∈I Ci = C.
Alternatively, apply Corollary 5.18. ⊓
⊔

5.3 Iterating Compositions of Averaged Operators

Our first result concerns the asymptotic behavior of iterates of a composition

of averaged nonexpansive operators with possibly no common fixed point.

Theorem 5.22 Let D be a nonempty weakly sequentially closed (e.g., closed

and convex) subset of H, let m be a strictly positive integer, set I =
5.3 Iterating Compositions of Averaged Operators 83

{1, . . . , m}, let (Ti )i∈I be a family of nonexpansive operators from D to D

such that Fix(T1 · · · Tm ) 6= ∅, and let (αi )i∈I be real numbers in ]0, 1[ such
that, for every i ∈ I, Ti is αi -averaged. Let x0 ∈ D and set

(∀n ∈ N) xn+1 = T1 · · · Tm xn . (5.20)

Then xn − T1 · · · Tm xn → 0, and there exist points y1 ∈ Fix T1 · · · Tm , y2 ∈

Fix T2 · · · Tm T1 , . . . , ym ∈ Fix Tm T1 · · · Tm−1 such that

xn ⇀ y1 = T1 y2 , (5.21)
Tm xn ⇀ ym = Tm y1 , (5.22)
Tm−1 Tm xn ⇀ ym−1 = Tm−1 ym , (5.23)
..
.
T3 · · · Tm xn ⇀ y3 = T3 y4 , (5.24)
T2 · · · Tm xn ⇀ y2 = T2 y3 . (5.25)

Proof. Set T = T1 · · · Tm and (∀i ∈ I) βi = (1 − αi )/αi . Now take y ∈ Fix T .

The equivalence (i)⇔(iii) in Proposition 4.25 yields

kxn+1 − yk2 = kT xn − T yk2

≤ kT2 · · · Tm xn − T2 · · · Tm yk2
− β1 k(Id − T1 )T2 · · · Tm xn − (Id − T1 )T2 · · · Tm yk2
≤ kxn − yk2 − βm k(Id − Tm )xn − (Id − Tm )yk2
− βm−1 k(Id − Tm−1 )Tm xn − (Id − Tm−1 )Tm yk2 − · · ·
− β2 k(Id − T2 )T3 · · · Tm xn − (Id − T2 )T3 · · · Tm yk2
− β1 k(Id − T1 )T2 · · · Tm xn − (T2 · · · Tm y − y)k2 . (5.26)

Therefore, (xn )n∈N is Fejér monotone with respect to Fix T and

(Id − Tm )xn − (Id − Tm )y → 0, (5.27)

(Id − Tm−1 )Tm xn − (Id − Tm−1 )Tm y → 0, (5.28)
..
.
(Id − T2 )T3 · · · Tm xn − (Id − T2 )T3 · · · Tm y → 0, (5.29)
(Id − T1 )T2 · · · Tm xn − (T2 · · · Tm y − y) → 0. (5.30)

Upon adding (5.27)–(5.30), we obtain xn − T xn → 0. Hence, since T is

nonexpansive as a composition of nonexpansive operators, it follows from
Theorem 5.13(i) that (xn )n∈N converges weakly to some point y1 ∈ Fix T ,
which provides (5.21). On the other hand, (5.27) yields Tm xn − xn →
Tm y1 − y1 . So altogether Tm xn ⇀ Tm y1 = ym , and we obtain (5.22). In
turn, since (5.28) asserts that Tm−1 Tm xn − Tm xn → Tm−1 ym − ym , we ob-
84 5 Fejér Monotonicity and Fixed Point Iterations

tain Tm−1 Tm xn ⇀ Tm−1 ym = ym−1 , hence (5.23). Continuing this process,

we arrive at (5.25). ⊓
⊔
As noted in Remark 5.19, results on averaged nonexpansive operators ap-
ply in particular to firmly nonexpansive operators and projectors onto convex
sets. Thus, by specializing Theorem 5.22 to convex projectors, we obtain the
iterative method described in the next corollary, which is known as the POCS
(Projections Onto Convex Sets) algorithm in the signal recovery literature.

Corollary 5.23 (POCS algorithm) Let m be a strictly positive integer,

set I = {1, . . . , m}, let (Ci )i∈I be a family of nonempty closed convex subsets
of H, let (Pi )i∈I denote their respective projectors, and let x0 ∈ H. Suppose
that Fix(P1 · · · Pm ) 6= ∅ and set

(∀n ∈ N) xn+1 = P1 · · · Pm xn . (5.31)

Then there exists (y1 , . . . , ym ) ∈ C1 × · · · × Cm such that xn ⇀ y1 = P1 y2 ,

Pm xn ⇀ym = Pm y1 , Pm−1 Pm xn ⇀ym−1 = Pm−1 ym , . . ., P3 · · · Pm xn ⇀y3 =
P3 y4 , and P2 · · · Pm xn ⇀ y2 = P2 y3 .

Proof. This follows from Proposition 4.8 and Theorem 5.22. ⊓

⊔

Remark 5.24 In Corollary 5.23, suppose that, for some j ∈ I, Cj is

bounded. Then Fix(P1 · · · Pm ) 6= ∅. Indeed, consider the circular composition
of the m projectors given by T = Pj · · · Pm P1 · · · Pj−1 . Then Proposition 4.8
asserts that T is a nonexpansive operator that maps the nonempty bounded
closed convex set Cj to itself. Hence, it follows from Theorem 4.19 that there
exists a point x ∈ Cj such that T x = x.

The next corollary describes a periodic projection method to solve a convex

feasibility problem.

Corollary 5.25 Let m be a strictly positive integer, set I = {1, T . . . , m}, let
(Ci )i∈I be a family of closed convex subsets of H such that C = i∈I Ci 6= ∅,
let (Pi )i∈I denote their respective projectors, and let x0 ∈ H. Set (∀n ∈ N)
xn+1 = P1 · · · Pm xn . Then (xn )n∈N converges weakly to a point in C.

Proof. Using Corollary 5.23, T

Proposition 4.8, and Corollary 4.37, we obtain
xn ⇀ y1 ∈ Fix(P1 · · · Pm ) = i∈I Fix Pi = C. Alternatively, this is a special
case of Example 5.20. ⊓
⊔

Remark 5.26 If, in Corollary 5.25, all the sets are closed affine subspaces,
so is C and we derive from Proposition 5.9(i) that xn ⇀ PC x0 . Corollary 5.28
is classical, and it states that the convergence is actually strong in this case.
In striking contrast, the example constructed in [146] provides a closed hy-
perplane and a closed convex cone in ℓ2 (N) for which alternating projections
converge weakly but not strongly.
Exercises 85

The next result will help us obtain a sharper form of Corollary 5.25 for
closed affine subspaces.

Proposition 5.27 Let T ∈ B(H) be nonexpansive and let x0 ∈ H. Set V =

Fix T and (∀n ∈ N) xn+1 = T xn . Then xn → PV x0 ⇔ xn − xn+1 → 0.

Proof. If xn → PV x0 , then xn − xn+1 → PV x0 − PV x0 = 0. Conversely,

suppose that xn − xn+1 → 0. We derive from Theorem 5.13(ii) that there
exists v ∈ V such that xn → v. In turn, Proposition 5.9(i) yields v = PV x0 .
⊓
⊔

Corollary 5.28 (von Neumann–Halperin) Let m be a strictly positive

integer, set I = {1,T. . . , m}, let (Ci )i∈I be a family of closed affine subspaces of
H such that C = i∈I Ci 6= ∅, let (Pi )i∈I denote their respective projectors,
let x0 ∈ H, and set

(∀n ∈ N) xn+1 = P1 · · · Pm xn . (5.32)

Then xn → PC x0 .

Proof. Set T = P1 · · · Pm . Then T is nonexpansive, and Fix T = C by Corol-

lary 4.37.
We first assume that each set Ci is a linear subspace. Then T is odd,
and Theorem 5.22 implies that xn − T xn → 0. Thus, by Proposition 5.27,
xn → PC x0 .
We now turn our attention to the general affine case. Since C 6= ∅, there
exists y ∈ C such that for every i ∈ I, Ci = y + Vi , i.e.,
T Vi is the closed linear
subspace parallel to Ci , and C = y + V , where V = i∈I Vi . Proposition 3.17
implies that, for every x ∈ H, PC x = Py+V x = y + PV (x − y) and (∀i ∈ I)
Pi x = Py+Vi x = y + PVi (x − y). Using these identities repeatedly, we obtain

(∀n ∈ N) xn+1 − y = (PV1 · · · PVm )(xn − y). (5.33)

Invoking the already verified linear case, we get xn − y → PV (x0 − y) and

conclude that xn → y + PV (x0 − y) = PC x0 . ⊓
⊔

Exercises

Exercise 5.1 Find a nonexpansive operator T : H → H that is not firmly

nonexpansive and such that, for every x0 ∈ H, the sequence (T n x0 )n∈N con-
verges weakly but not strongly to a fixed point of T .

Exercise 5.2 Construct a non-Cauchy sequence (xn )n∈N in R that is asymp-

totically regular, i.e., xn − xn+1 → 0.
86 5 Fejér Monotonicity and Fixed Point Iterations

Exercise 5.3 Find an alternative proof of Theorem 5.5 based on Corol-

lary 5.8 in the case when C is closed and convex.

Exercise 5.4 Let C be a nonempty subset of H and let (xn )n∈N be a se-
quence in H that is Fejér monotone with respect to C. Show that (xn )n∈N is
Fejér monotone with respect to conv C.

Exercise 5.5 Let T : H → H be a nonexpansive operator such that Fix T 6=

∅, and let (xn )n∈N be a sequence in H such that
(i) for every x ∈ Fix T , (kxn − xk)n∈N converges;
(ii) xn − T xn → 0.
Show that (xn )n∈N converges weakly to a point in Fix T .

Exercise 5.6 Find a nonexpansive operator T : H → H that is not firmly

nonexpansive and such that, for every x0 ∈ H, the sequence (T n x0 )n∈N con-
verges weakly but not strongly to a fixed point of T .

Exercise 5.7 Let m be a strictly positive integer, set I = {1, T. . . , m}, let
(Ci )i∈I be a family of closed convex subsets of H such that C = i∈I Ci 6= ∅,
and let (Pi )i∈I be their respective projectors. Derive parts (ii) and (iii) from
(i) and Theorem 5.5, and also from Corollary 5.18.
(i) Let i ∈ I, let x ∈ Ci , and let y ∈ H. Show that kPi y − xk2 ≤ ky − xk2 −
kPi y − yk2 .
(ii) Set x0 ∈ H and
1

(∀n ∈ N) xn+1 = P1 xn + P1 P2 xn + · · · + P1 · · · Pm xn . (5.34)
m
(a) Let x P∈ C and n ∈ N. Show that kxn+1 − xk2 ≤ kxn − xk2 −
(1/m) i∈I kPi xn − xk2 .
(b) Let x be a weak sequential cluster point of (xn )n∈N . Show that
x ∈ C.
(c) Show that (xn )n∈N converges weakly to a point in C.
(iii) Set x0 ∈ H and
1

(∀n ∈ N) xn+1 = P1 P2 xn +P2 P3 xn +· · ·+Pm−1 Pm xn . (5.35)
m−1
(a) Let x ∈ C and n ∈ N. Show that kxn+1 − xk2 ≤ kxn − xk2 −
Pm−1 2 2
i=1 (kPi+1 xn − xn k + kPi Pi+1 xn − Pi+1 xn k )/(m − 1).
(b) Let x be a weak sequential cluster point of (xn )n∈N . Show that
x ∈ C.
(c) Show that (xn )n∈N converges weakly to a point in C.
Index

3∗ monotone, 354–356 base of neighborhoods, 23

best approximation, 44, 278, 410
accretive operator, 293 best approximation algorithm,
acute cone, 105 410, 411, 431
addition of functions, 6 biconjugate function, 181, 185,
addition of set-valued operators, 2 190, 192
adjoint of a linear operator, 31 biconjugation theorem, 190
affine constraint, 386 bilinear form, 39, 384, 385
affine hull, 1, 93 Bishop–Phelps theorem, 107
affine minorant, 133, 134, 168, 184, Boltzmann–Shannon entropy, 137,
191, 192, 207, 223, 224 139
affine operator, 3, 35, 44, 418 boundary of a set, 7
affine projector, 48, 62 bounded below, 134, 184
affine subspace, 1, 43, 108 Brézis–Haraux theorem, 358
almost surely, 29 Bregman distance, 258
alternating minimizations, 162 Browder–Göhde–Kirk theorem, 64
alternating projection method, Brøndsted–Rockafellar theorem,
406, 432 236
Anderson–Duffin theorem, 361 Bunt–Motzkin theorem, 318
angle bounded operator, 361, 362 Burg’s entropy, 138, 151, 244
Apollonius’s identity, 30, 46
approximating curve, 64 Cantor’s theorem, 17
asymptotic center, 117, 159, 195 Cauchy sequence, 17, 24, 34, 46
asymptotic regularity, 78 Cauchy–Schwarz inequality, 29
at most single-valued, 2 Cayley transform, 349
Attouch–Brézis condition, 210, chain, 3
216 chain rule, 40
Attouch–Brézis theorem, 207, 209 characterization of minimizers,
autoconjugate function, 190, 239, 381
303 Chebyshev center, 249, 250
averaged nonexpansive operator, Chebyshev problem, 47
67, 72, 80–82, 294, 298, 440 Chebyshev set, 44–47, 318
closed ball, 16
Baillon–Haddad theorem, 270 closed convex hull, 43
ball, 43 closed range, 220
Banach–Alaoglu theorem, 34 closed set, 7, 8
Banach–Picard theorem, 20 closure of a set, 7, 22
Banach–Steinhaus theorem, 31 cluster point, 7, 8
barrier cone, 103, 156
base of a topology, 7, 9, 16, 33

460
INDEX 461

cocoercive operator, 60, 61, 68, 70, cyclically monotone operator, 326
270, 294, 298, 325, 336, 339,
355, 370, 372, 377, 379, 438 Debrunner–Flor theorem, 315
coercive function, 158–161, 165, decreasing function, 5
202–204, 210 decreasing sequence of convex sets,
cohypomonotone operator, 337 48, 417
common fixed points, 71 demiclosedness principle, 63
compact set, 7, 8, 13 dense hyperplane, 123
complementarity problem, 376 dense set, 7, 33, 123, 232
complementary slackness, 291, 422 descent direction, 248, 249
complete metric space, 17 descent lemma, 270
composition of set-valued opera- diameter of a set, 16
tors, 2 directed set, 3, 4, 22, 27
concave function, 113 directional derivative, 241, 247
cone, 1, 87, 285, 287, 376, 387, 389, discontinuous linear functional,
425 32, 123, 169
conical hull, 87 discrete entropy, 140
conjugate function, 181 distance, 27
conjugation, 181, 197, 226, 230 distance to a set, 16, 20, 24, 32, 34,
constrained minimization prob- 44, 49, 98, 167, 170, 173, 177,
lem, 283, 285, 383 183, 185, 188, 238, 271, 272
continuity, 9 domain of a function, 5, 6, 113
continuous affine minorant, 168 domain of a set-valued operator, 2
continuous convex function, 123, domain of continuity, 11
136 Douglas–Rachford algorithm, 366,
continuous function, 11 376, 401, 404
continuous linear functional, 31 dual cone, 96
continuous operator, 9, 63 dual optimal value, 214
convergence of a net, 7 dual problem, 212, 214, 275, 279,
convex combination, 44 408
convex cone, 87, 89, 179, 183, 285, dual solution, 275, 279
425 duality, 211, 213, 275
convex feasibility problem, 81, 84 duality gap, 212, 214–216, 221
convex function, 113, 155 Dykstra’s algorithm, 431, 432
convex hull, 43, 44
Eberlein–Šmulian theorem, 35
convex integrand, 118, 138, 193,
effective domain of a function, 6
238
Ekeland variational principle, 19
convex on a set, 114, 125
Ekeland–Lebourg theorem, 263
convex programming problem, 290
enlargement of a monotone opera-
convex set, 43
tor, 309
convexity with respect to a cone,
entropy of a random variable, 139
285
epi-sum, 167
core, 90, 95, 123, 207, 210, 214,
epigraph, 5, 6, 12, 15, 113, 119,
216, 241, 271
133, 168
counting measure, 140
equality constraint, 283
462 INDEX

Euclidean space, 28 Fréchet gradient, 38

even function, 186 Fréchet topological space, 23
eventually in a set, 4 frequently in a set, 4
evolution equation, 313 function, 5
exact infimal convolution, 167,
170, 171, 207, 209, 210 Gδ set, 19, 263, 320
exact infimal postcomposition, 178 Gâteaux derivative, 37
exact modulus of convexity, 144– Gâteaux differentiability, 37–39,
146 243, 244, 246, 251, 252, 254,
existence of minimizers, 157, 159 257, 267
expected value, 29 gauge, 120, 124, 202
extended real line, 4 generalized inverse, 50, 251, 360,
extension, 297 361, 395, 418
generalized sequence, 4
Fσ set, 24 global minimizer, 223
Farkas’s lemma, 99, 106 gradient, 38, 176, 243, 244, 266,
farthest-point operator, 249, 296 267, 382
Fejér monotone, 75, 83, 86, 160, gradient operator, 38
400 graph, 5
Fenchel conjugate, 181 graph of a set-valued operator, 2
Fenchel duality, 211
Fenchel–Moreau theorem, 190 Hölder continuous gradient, 269
Fenchel–Rockafellar duality, 213, half-space, 32, 33, 43, 419, 420
275, 282, 408 Hamel basis, 32
Fenchel–Young inequality, 185, hard thresholder, 61
226 Haugazeau’s algorithm, 436, 439
Fermat’s rule, 223, 235, 381 Hausdorff distance, 25
firmly nonexpansive operator, 59, Hausdorff space, 7, 16, 33
61–63, 68, 69, 73, 80, 81, 176, hemicontinous operator, 298, 325
270, 294, 298, 335, 337, 436 Hessian, 38, 243, 245, 246
first countable space, 23 Hilbert direct sum, 28, 226
Fitzpatrick function, 304, 311, 351 Hilbert space, 27
Fitzpatrick function of order n, Huber’s function, 124
330 hyperplane, 32, 34, 48, 123
fixed point, 62, 79–81
fixed point iterations, 75 increasing function, 5
fixed point set, 20, 62–64, 436 increasing sequence of convex sets,
forward–backward algorithm, 370, 416
377, 405, 438, 439 indicator function, 12, 113, 173,
forward–backward–forward algo- 227
rithm, 375 inequality constraint, 285, 389
Fréchet derivative, 38, 39, 257 infimal convolution, 167, 187, 207,
Fréchet differentiability, 38, 176, 210, 237, 266, 359
177, 243, 253, 254, 268–270, infimal postcomposition, 178, 187,
320 199, 218, 237
infimum, 5, 157, 159, 184, 188
INDEX 463

infimum of a function, 6 Lipschitz continuous, 20, 31, 59,

infinite sum, 27 123, 176, 229, 339
initial condition, 295 Lipschitz continuous gradient,
integral function, 118, 138, 193, 269–271, 405–407, 439
238 local minimizer, 156
interior of a set, 7, 22, 90, 123 locally bounded operator, 316,
inverse of a monotone operator, 319, 344
295 locally Lipschitz continuous, 20,
inverse of a set-valued operator, 2, 122
231 lower bound, 3
inverse strongly monotone opera- lower level set, 5, 6, 148, 427
tor, 60 lower semicontinuity, 10, 129
lower semicontinuous, 10, 12
Jensen’s inequality, 135 lower semicontinuous convex enve-
lope, 130, 185, 192, 193, 207
Karush–Kuhn–Tucker conditions, lower semicontinuous convex func-
393 tion, 122, 129, 132, 185
Kenderov theorem, 320 lower semicontinuous envelope, 14,
kernel of a linear operator, 32 23
Kirszbraun–Valentine theorem, lower semicontinuous function,
337 129
Krasnosel’skiı̆–Mann algorithm, lower semicontinuous infimal con-
78, 79 volution, 170, 210
Lagrange multiplier, 284, 287, 291, marginal function, 13, 120, 152
386–388, 391 max formula, 248
Lagrangian, 280, 282 maximal element, 3
Lax–Milgram theorem, 385 maximal monotone operator, 297
least element, 3 maximal monotonicity and conti-
least-squares solution, 50 nuity, 298
Lebesgue measure, 29 maximal monotonicity of a sum,
Legendre function, 273 351
Legendre transform, 181 maximally cyclically monotone op-
Legendre–Fenchel transform, 181 erator, 326
level set, 5, 6, 12, 15, 132, 158, 203, maximally monotone extension,
383 316, 337
limit inferior, 5 maximally monotone operator,
limit superior, 5 297, 298, 311, 335, 336, 338,
line segment, 1, 43, 54, 132 339, 438
linear convergence, 20, 78, 372, maximum of a function, 6
377, 406, 407 measure space, 28, 295
linear equations, 50 metric space, 16
linear functional, 32 metric topology, 16, 33, 34
linear monotone operator, 296– metrizable topology, 16, 23, 34
298, 355 midpoint convex function, 141
464 INDEX

midpoint convex set, 57 obtuse cone, 105

minimax, 218 odd operator, 79, 379
minimization in a product space, open ball, 16
403 open set, 7
minimization problem, 13, 156, operator splitting algorithm, 366
381, 393, 401, 402, 404–406 Opial’s condition, 41
minimizer, 156, 157, 159, 163, 243, optimal value, 214
384 order, 3
minimizing sequence, 6, 13, 160, orthogonal complement, 27
399 orthonormal basis, 27, 37, 161,
minimum of a function, 6, 243 301, 313, 344
Minkowski gauge, 120, 124, 202 orthonormal sequence, 34
Minty’s parametrization, 340 outer normal, 32
Minty’s theorem, 311
modulus of convexity, 144 parallel linear subspace, 1
monotone extension, 297 parallel projection algorithm, 82
monotone linear operator, 296 parallel splitting algorithm, 369,
monotone operator, 244, 293, 311, 404
351, 363 parallel sum of monotone opera-
monotone set, 293 tors, 359
Moore–Penrose inverse, 50 parallelogram identity, 29
Moreau envelope, 173, 175, 176, parametric duality, 279
183, 185, 187, 197, 198, 270, paramonotone operator, 323, 385
271, 276, 277, 334, 339, 342 partial derivative, 259
Moreau’s conical decomposition, partially ordered set, 3
98 Pasch–Hausdorff envelope, 172,
Moreau’s decomposition, 198 179
Moreau–Rockafellar theorem, 204 periodicity condition, 295
perspective function, 119, 184
negative orthant, 5 POCS algorithm, 84
negative real number, 4 pointed cone, 88, 105
neighborhood, 7 pointwise bounded operator fam-
net, 4, 5, 22, 27, 53, 314 ily, 31
nonexpansive operator, 59, 60, 62, polar cone, 96, 110
63, 79, 159, 270, 294, 298, polar set, 110, 202, 206, 428
336, 348, 439 polarization identity, 29
nonlinear equation, 325 polyhedral cone, 388, 389
norm, 27, 35, 40, 115, 118, 144, polyhedral function, 216–218, 381,
147, 150, 151, 183, 199, 231, 383, 388, 389
252 polyhedral set, 216, 383
norm topology, 33 polyhedron, 419
normal cone, 101, 227, 230, 238, positive operator, 60
272, 304, 334, 354, 383, 389 positive orthant, 5, 426
normal equation, 50 positive real number, 4
normal vector, 107 positive semidefinite matrix, 426
INDEX 465

positively homogeneous function, proximal-point algorithm, 345,

143, 201, 229, 278 399, 438
positively homogeneous operator, proximinal set, 44–46
3 proximity operator, 175, 198, 199,
power set, 2 233, 243, 244, 271, 334, 339,
primal optimal value, 214 342–344, 375, 381, 382, 401,
primal problem, 212, 214, 275, 279, 402, 404, 405, 415, 428
408 pseudocontractive operator, 294
primal solution, 275, 279 pseudononexpansive operator, 294
primal–dual algorithm, 408
probability simplex, 426 quadratic function, 251
probability space, 29, 139 quasiconvex function, 148, 157,
product topology, 7 160, 165
projection algorithm, 431, 439 quasinonexpansive operator, 59,
projection onto a ball, 47 62, 71, 75
projection onto a convex cone, 97, quasirelative interior, 91
98, 425
projection onto a half-space, 419 random variable, 29, 135, 139, 194
projection onto a hyperplane, 48, range of a set-valued operator, 2
419 range of a sum of operators, 357,
projection onto a hyperslab, 419 358
projection onto a linear subspace, recession cone, 103
49 recession function, 152
projection onto a lower level set, recovery of primal solutions, 275,
427 408
projection onto a polar set, 428 reflected resolvent, 336, 363, 366
projection onto a ray, 426 regularization, 393
projection onto a set, 44 regularized minimization problem,
projection onto an affine subspace, 393
48, 77, 417 relative interior, 90, 96, 123, 210,
projection onto an epigraph, 133, 216, 234
427 resolvent, 333, 335, 336, 366, 370,
projection operator, 44, 61, 62, 373
175, 177, 334, 360, 361, 415 reversal of a function, 186, 236,
projection theorem, 46, 238 342
projection-gradient algorithm, 406 reversal of an operator, 3, 340
projector, 44, 61 Riesz–Fréchet representation, 31
proper function, 6, 132 right-shift operator, 330, 356
proximal average, 199, 205, 271, Rådström’s cancellation, 58
307
saddle point, 280–282
proximal mapping, 175
scalar product, 27
proximal minimization, 399
second Fréchet derivative, 38
proximal-gradient algorithm, 405,
second Gâteaux derivative, 38
439
second-order derivative, 245, 246
466 INDEX

selection of a set-valued operator, strictly quasiconvex function, 149,

2 157
self-conjugacy, 183, 185 strictly quasinonexpansive opera-
self-dual cone, 96, 186 tor, 59, 71
self-polar cone, 186 string-averaged relaxed projec-
separable Hilbert space, 27, 194 tions, 82
separated sets, 55 strong convergence, 33, 37
separation, 55 strong relative interior, 90, 95, 96,
sequential cluster point, 7, 15, 33 209, 210, 212, 215, 217, 234,
sequential topological space, 16, 23 236, 381
sequentially closed, 15, 16, 53, 231, strong separation, 55
300, 301 strong topology, 33
sequentially compact, 15, 16, 36 strongly convex function, 144, 159,
sequentially continuous, 15, 16 188, 197, 270, 276, 324, 406
sequentially lower semicontinuous, strongly monotone operator, 323,
15, 129 325, 336, 344, 372
set-valued operator, 2 subadditive function, 143
shadow sequence, 76 subdifferentiable function, 223,
sigma-finite measure space, 194 247
Slater condition, 391 subdifferential, 223, 294, 304, 312,
slope, 168 324, 326, 354, 359, 381, 383
soft thresholder, 61, 199 subdifferential of a maximum, 264
solid cone, 88, 105 subgradient, 223
span of a set, 1 sublinear function, 143, 153, 156,
splitting algorithm, 375, 401, 402, 241
404, 405 subnet, 4, 8, 22
Stampacchia’s theorem, 384, 395 sum of linear subspaces, 33
standard unit vectors, 28, 89, 92 sum of monotone operators, 351
steepest descent direction, 249 sum rule for subdifferentials, 234
strict contraction, 64 summable family, 27
strict epigraph, 180 supercoercive function, 158, 159,
strictly convex function, 114, 144, 172, 203, 210, 229
161, 267, 324 support function, 109, 156, 183,
strictly convex on a set, 114 195, 201, 229, 240
strictly convex set, 157 support point, 107, 109, 164
strictly decreasing function, 5 supporting hyperplane, 107, 109
strictly increasing function, 5 supremum, 5, 129, 188
strictly monotone operator, 323, supremum of a function, 6
344 surjective monotone operator, 318,
strictly negative real number, 4 320, 325, 358
strictly nonexpansive operator,
325 tangent cone, 100
strictly positive operator, 246 time-derivative operator, 295, 312,
strictly positive orthant, 5 334
strictly positive real number, 4 Toland–Singer duality, 205
INDEX 467

topological space, 7 weakly lower semicontinuous, 35,

topology, 7 129
totally ordered set, 3 weakly lower semicontinuous func-
trace of a matrix, 28 tion, 33
translation of an operator, 3 weakly open, 33
Tseng’s splitting algorithm, 373, weakly sequentially closed, 33–35,
378, 407 53
Tykhonov regularization, 393 weakly sequentially compact, 33,
35
unbounded net, 314 weakly sequentially continuous op-
uniform boundedness principle, 31 erator, 343, 426
uniformly convex function, 144, weakly sequentially lower semicon-
147, 324, 394, 399 tinuous, 129
uniformly convex on a set, 144, Weierstrass theorem, 13
147, 324, 407
uniformly convex set, 164, 165 Yosida approximation, 333, 334,
uniformly monotone on a set, 324 336, 339, 345, 347, 348
uniformly monotone on bounded
sets, 346, 367, 373, 376, 378, zero of a monotone operator, 344,
408 345, 347, 381, 438
uniformly monotone operator, zero of a set-valued operator, 2
323, 325, 344, 354, 358, 367, zero of a sum of operators, 363,
373, 376, 378, 408 366, 369, 375
uniformly quasiconvex function,
149, 163
upper bound, 3
upper semicontinuous function,
11, 124, 281

value function, 279, 289

variational inequality, 375–378,
383
Volterra integration operator, 308
von Neumann’s minimax theorem,
218
von Neumann–Halperin theorem,
85

weak closure, 53
weak convergence, 33, 36, 79–81
weak sequential closure, 53
weak topology, 33
weakly closed, 33–35, 45, 53
weakly compact, 33–35
weakly continuous operator, 33,
35, 62, 418