Scribd
Upload
53 views10 pages

CPSC 542f Notes

The document discusses convex sets and convex functions. It defines key concepts such as convexity, cones, epigraphs, support functions, and operations that preserve convexity. Convex sets are sets where the line segment between any two points will always lie within the set. Convex functions have the property that for any two inputs, the line between the corresponding outputs will always lie below the line connecting the outputs. Support functions are used to represent convex sets and have properties like positive homogeneity and sub-linearity. Operations like positive affine transformations and taking the supremum of functions preserve convexity of functions.

Uploaded by

Sofia Vega
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
53 views10 pages

CPSC 542f Notes

The document discusses convex sets and convex functions. It defines key concepts such as convexity, cones, epigraphs, support functions, and operations that preserve convexity. Convex sets are sets where the line segment between any two points will always lie within the set. Convex functions have the property that for any two inputs, the line between the corresponding outputs will always lie below the line connecting the outputs. Support functions are used to represent convex sets and have properties like positive homogeneity and sub-linearity. Operations like positive affine transformations and taking the supremum of functions preserve convexity of functions.

Uploaded by

Sofia Vega
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

Contents

1 Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1 Convexity 5
1.1.1 Cone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Convex Functions 6
1.2.1 Epigraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Support Function 8
1.4 Operations Preserve Convexity of Functions 9
1. Convex Sets

1.1 Convexity
1.1.1 Cone
Definition 1.1.1 — Cone. A set K ∈ Rn , when x ∈ K implies αx ∈ K.

A non convex cone can be hyper-plane.


For convex cone x + y ∈ K, ∀x, y ∈ K.
Cone don’t need to be "pointed". e.g.
Direct sums of cones C1 +C2 = {x = x1 + x2 |x1 ∈ C1 , x2 ∈ C2 }.

■ Example 1.1 S1n {X|X = X n , λ (x) ≥ 0}


A matrix with positive eigenvalues. ■

Operations preserving convexity


Intersection C ∩i∈I Ci
Linear map Let A : Rn → Rn be a linear map. If C ∈ Rn is convex, so is A(C) = {Ax∀x ∈ C}
Inverse image A−1 (D) = {x ∈ R|Ax ∈ D}

Operations that induce convexity


Convex hull on S = ∩{C|S ∈ C,Cisconvex}

■ Example 1.2 Co{x1 , x2 , · · · , xm } = {∑m


i=1 αi xi |α ∈ δm } ■

For a convex set x ∈ C ⇒ x = ∑ αi xi .

Theorem 1.1.1 — Carathéodory’s theorem. If a point x ∈ Rd lies in the convex hull of a set P,
there is a subset P′ of P consisting of d + 1 or fewer points such that x lies in the convex hull of P′ .
Equivalently, x lies in an r-simplex with vertices in P.
6 Chapter 1. Convex Sets

1.2 Convex Functions


Definition 1.2.1 — Convex function. Let C ∈ Rn be convex, f : C → R is convex on f if
x, y ∈ C ×C. ∀α ∈ (0, 1), f (αx + (1 − α)y) ≤ f (αx) + f ((1 − α)y)

Definition 1.2.2 — Strictly Convex function. Let C ∈ Rn be convex, f : C → R is strictly convex


on f if x, y ∈ C ×C. ∀α ∈ (0, 1), f (αx + (1 − α)y)⟨ f (αx) + f ((1 − α)y)

Definition 1.2.3 — Strongly convex. f : C → R is strongly convex with modules u ≥ 0 if


f − 21 u|| · ||2 is convex.

Interpretation: There is a convex quadratic 21 u|| · ||2 that lower bounds f.


■ Example 1.3 minx∈C f (x) ↔ min f¯(x) Useful to turn this into an unconstrained problem.

(
f (x) i f x ∈ C
f¯(x) =
∞ elsewhere
■

Definition 1.2.4 A function f : Rn → R ∪ ∞ R̄ is convex if x, y ∈ Rn × Rn , ∀x, y, f¯(αx + (1 −


α)y) ≤ f (αx) + f ((1 − α)y)

Definition 1 is equivalent to definition 2 if f (x) = ∞.


■ Example 1.4 f (x) = sup j∈J f j (x) ■

1.2.1 Epigraph
Definition 1.2.5 — Epigraph. For f : Rn → R̄, its epigraph epi( f ) ∈ Rn+1 isthesetepi( f ){(x, α)| f (x) ∈
α}

Next: a function is convex i.f.f. its epigraph is convex.


Definition 1.2.6 A function f : C → R,C ∈ Rn is convex if ∀x, y ∈ C, f (ax + (1 − a)x) ≤ a f (x) +
(1 − a) f (x) ∀a ∈ (0, 1).
Strict convex: x ̸= y ⇒ f (ax + (1 − a)x) ≤ a f (x) + (1 − a) f (x)

R f is convex ⇒ − f is concave.

Level set: Sα f = {x| f (x) ≤ α}.


Sα f is convex ⇔ f is convex.

Definition 1.2.7 — Strongly convex. f : C → R is strongly convex with modules µ if ∀x, y ∈ C,


1
∀α ∈ (0, 1), f (ax + (1 − a)x) ≤ a f (x) + (1 − a) f (x) − 2µ α(1 − α)∥x − y∥2 .

R
• f is 2nd-differentiable, f ix convex ⇐⇒ ∇2 f (x)⟩0.
• f is strongly convex ⇐⇒ ∇2 f (x)⟩µI ⇐⇒ x ≥ µ
1.2 Convex Functions 7

Definition 1.2.8 — 2. f : Rn → R̄ is convexif x, y ∈ R, α ∈ (0, 1), f (ax + (1 − a)x) ≤ a f (x) +


(1 − a) f (x).

The effective domain of f is dom f = {x| f (x)⟨+∞}


(
0 x ∈C
■ Example 1.5 — ludcator function. δc (x) = .
+∞ elsewhere
domδc (x) = C ■

Definition 1.2.9 — Epigraph. The epigraph of f is epi f = {(x, α)| f (x) ≤ α}

The graph of epi f is {(x, f (x)|x ∈ dom f }.


Definition 1.2.10 — III. A function f : Rn → R̄ is

Theorem 1.2.1 f : Rn → R̄ is convex ⇐⇒ ∀x, y ∈ Rn , α ∈ (0, 1), f (ax + (1 − a)x) ≤ a f (x) +


(1 − a) f (x).

Proof. ⇒ take x, y ∈ dom f , (x, f (x)) ∈ epi f ,(y, f (y)) ∈ epi f . ■

■ Example 1.6 — Distance. Distance to a convexset dc (x) = inf{∥z − x∥|z ∈ C}. Take any two

sequence {yk }and{ȳk } ⊂ C s.t. ∥yk − x∥ → dc (x), ∥ȳk − x̄∥ → dc (x̄). zk = αyk + (1 − α)ȳk .

dc (αx + (1 − α)x̄) ≤ ∥zk − αx − (1 − α)x̄∥


= ∥α(yk − x) + (1 − α)(ȳk − x̄)∥
≤ α∥yk − x∥ + (1 − α)∥ȳk − x̄∥

Take k → ∞, dc (αx + (1 − α)x̄) ≤ αd(x) + (1 − α)d(x̄) ■

■ Example 1.7 — Eigenvalues. Let X ∈ Sn := {n × nsymmetricmatrix}. λ1 (x) ≥ λ2 (X) ≥ . . . ≥


λn (x).
fk (x) = ∑n1 λi (x).
Equivalent characterization

fk (x) = max{∑ vTi Xvi |vi ⊥ v j , i ̸= j}


i
= max{tr(V T XV |V T V = Ik }
max{tr(VV T X)}by circularity

Note ⟨A, B⟩ = tr(A, B) is true for symmetric matrix.


⟨A, A⟩ = |A|2F = ∑i A2ii ■
8 Chapter 1. Convex Sets

1.3 Support Function

Take a set C ∈ Rn , not necessarily convex.The support function is σC = Rn → R̄. σC (x) =

∑{⟨x, u⟩|u ∈ C}.


Fact 1.3.1 The support function binds the supporting hyper-plane.

Supporting functions are

• Positively homogeneous
σC (αx) = ασC (x)∀α⟩0
σC (αx) = supu∈C ⟨αx, u⟩ = α supu∈C ⟨x, u⟩ = ασC (x)
• Sub-linear( a special case of convex, linear combination holds ∀α.
σC (αx + (1 − α)y) = supu∈C ⟨αx + (1 − α)y, u⟩ ≤ α supu∈C ⟨x, u⟩ + (1 − α) supu∈C ⟨y, u⟩

■ Example 1.8 — L2-norm. ∥x∥ = supu∈C {⟨x, u⟩, u ∈ Rn }.


1
∥x∥ p = sup{⟨x, u⟩, u ∈ Bq } where p + 1q = 1. Bq = {∥x∥q ≤ 1}.
The norm is

• Positive homogeneous
• sub-linear
• If 0 ∈ C, σC is non-negative.
• If C is central-symmetric, σC (0) = 0 and σC (x) = σC (−x)

Fact 1.3.2 — Epigraph of a support function. epiσC = {(x,t)|σC (x) ≤ t}. Suppose (x,t) ∈ epiσC .
Take any α > 0. α(x,t) = (αx, αt).
ασC (x) = ασC (x) ≤ αt. α(x, c) ∈ epiσC
1.4 Operations Preserve Convexity of Functions 9

1.4 Operations Preserve Convexity of Functions


• Positive affine transformation
f1 , f2 , . . . , fk ∈ cvxRn .
f = α1 f1 + α2 f2 + . . . + αk fk
• Supremum of functions. Let { fi }i∈I be arbitrary family of functions. If ∃x sup j∈J f j (x) < ∞ ⇔
f (x) = sup j∈J f j (x)

• Composition with linear map.


f ∈ cvxRn , A : Rn → Rm is a linear map. f ◦ A(x) = f (Ax) ∈ cvxRn
10 Chapter 1. Convex Sets

f ◦ A(x) = f (A(αx + (1 − α)y))


= f (Aαx + (1 − α)Ay)
≤ α f (Ax) + (a − α) f (Ay)

jkdjcsudc ■

You might also like