Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

SC 639 (Spring 2020) - Mathematical Structures for Control

Ravi N. Banavar 1
Ashutosh Jindal 1

1
Systems and Control Engineering,
IIT Bombay, India

July 30, 2020

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Outline

1 Partial Derivatives

2 Differentiablity

3 Derivatives in Vector Space

4 Inverse Function Theorem

5 Implicit Function Theorem

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Outline

1 Partial Derivatives

2 Differentiablity

3 Derivatives in Vector Space

4 Inverse Function Theorem

5 Implicit Function Theorem

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Partial Derivatives

Definition
Let U be an open subset of Rn , and let f : U 7→ Rn . We define partial
derivative at a point x ∈ U by
f (x + hei ) − f (x)
Di f (x) = lim
h→0 h
f (x1 , . . . , xi + h, . . . , xn ) − f (x1 , . . . , xn )
= lim
h→0 h
If this limit exists, Di f (x) is the partial derivative with respect to xi
evaluated at x and is denoted by
∂f
Di f (x) = (x)
∂xi

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Outline

1 Partial Derivatives

2 Differentiablity

3 Derivatives in Vector Space

4 Inverse Function Theorem

5 Implicit Function Theorem

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Differentiable Function
A function f : U → R is said to be differentiable at a point x ∈ U if there
exist a vector A ∈ Rn and h ∈ Rn and a function ψ : Rn → R such that

f (x + h) = f (x) + hA, hi + khk ψ(h)

and ψ satisfies
lim ψ(h) = 0
h→0

Gradient
Let f (·) : U → R, and all partial derivatives Di f exist at x ∈ U then the
gradient is defined by
 
∂f ∂f
∇ f (x) := (x), . . . , (x)
∂x1 ∂xn

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Gradient and Differentiability

f : U → R is differentiable at a point x ∈ U if and only if ∇ f (x) exists at x
and
A = ∇ f (x)

Chain Rule
Let ϕ : J → Rn be a differentiable function defined on some interval J and
values in some open set U ⊂ Rn . Let be f : U → Rn a differentiable
function, then f ◦ ϕ : J → R is differentiable and for a given t ∈ J

(f ◦ ϕ)0 (t) = ∇ f (ϕ(t)).ϕ0 (t)

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Gradient and Tangent Space

Directional Derivative
Let x be a point of U and let v be a fixed vector with kvk = 1. Directional
derivative of f at x in the direction of v is given by

d
Dv f (x) = f (x + tv) = h∇ f (x), vi
dt
t=0

Direction of ∇ f (x) is the direction of maximal increase of the function f at

x

Hypersurface and Tangent Hyperplane

Let f be a differentiable function on some open set U in Rn . Let c ∈ R and
let S be the set of points x such that

f (x) = c, but ∇ f (x) 6= 0.

The set S is called a hypersurface in Rn . The tangent hyperplane of S at a

point p ∈ S is defined as the hyperplane passing through p and
perpendicular to ∇ f (p).

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

∇𝑓(𝑝)

Tangent hyperplane
𝑝 at 𝑝

𝑆 = 𝑥 ∈ 𝑈 𝑓 𝑥 = 0}

Level Surface S and the gradient ∇ f (p) at a point p ∈ S

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Outline

1 Partial Derivatives

2 Differentiablity

3 Derivatives in Vector Space

4 Inverse Function Theorem

5 Implicit Function Theorem

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Derivatives as a Linear Map

Space of Continuous Linear Maps

Let E, F be normed vector spaces. We denote the space of continuous
linear maps λ : E → F by L(E, F ). L(E, F ) assumes a vector space
structure. For λ1 , λ2 ∈ L(E, F ) and c ∈ R

(λ1 + λ2 )(x) = λ1 (x) + λ2 (x)

(cλ)(x) = cλ(x)

Derivative as a Linear Map

Let U be open in E, and let x ∈ U . Let f : U → F be a map. The f is said
to be differentiable at x if there exists a continuous linear map λ : E → F
and a map ψ defined for all sufficiently small h ∈ E and values in F , such
that
lim ψ(h) = 0
h→0

and
f (x + h) = f (x) + λ(h) + khk ψ(h)

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Jacobian Matrix
Let U be an open set of Rn , and f : U → Rm be a differentiable map at x.
Then the continuous linear map is represented by the matrix
∂f
Df (x) = (x)
∂x
and the matrix  ∂f 
1 ∂f1
(x)
∂x1
... ∂xn
(x)
 . .. 
Df (x) =  ..

.


∂fm ∂fm
∂x1
(x) ... ∂xn
(x)
is called the Jacobian of f at x.

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Properties of the Derivative

• Sum : Let U ⊂ E be a open set and f, g : U → F be differentiable at

x ∈ U . Then f + g is differentiable at x and

(f + g)0 (x) = f 0 (x) + g 0 (x)

and for c ∈ R
(cf )0 (x) = cf 0 (x)
• Product : Let f, g : U → F be differentiable at x ∈ U . Then the
product map f g is differentiable at x, and

(f g)0 (x) = f 0 (x)g(x) + f (x)g 0 (x)

• Chain Rule : Let U be open in E and V be open in F . Let

f : U → V and g : V → G be functions differentiable at x ∈ U and
f (x) ∈ V respectively. Then g ◦ f is differentiable at x and

(g ◦ f )0 (x) = g 0 (f (x)) ◦ f 0 (x)

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Second Derivative

Definition
Let U be open in E and let f : U → F be a differentiable map. Then if it
exists, the second derivative is a map defined as

D2 f = f (2) : U → L(E, L(E, F )) (1)

Theorem
Let U be open in E and f : U → F be twice differentiable and such that D2 f
is continuous. Then for each x ∈ U the bilinear map D2 f is symmetric i.e.

D2 f (x)(v, w) = D2 f (x)(w, v)

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Hessian of f

Theorem
Let U be open in Rn and let f : U → R be a function. Then f is of class C 2
if and only if all partial derivatives of f upto order ≤ 2 exists and are
continuous.

Hessian
The matrix representation of D2 f (x) is called the Hessian of f at x and is
denoted by  2 2

∂ f
∂x2
(x) . . . ∂x∂1 ∂x
f
n
(x)
 1 
∇2 f (x) = 
 .. .. 
 2 . . 

∂ f ∂2f
∂xn ∂x1
(x) . . . ∂x2
(x)
n

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Outline

1 Partial Derivatives

2 Differentiablity

3 Derivatives in Vector Space

4 Inverse Function Theorem

5 Implicit Function Theorem

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Inverse Function Theprem

Theorem
Inverse Function Theorem : Let U be open in E, let x0 ∈ U and let
f : U → F be a C 1 map. Assume the derivative f 0 (x0 ) : E → F is
invertible. Then f is locally C 1 -invertible at x0 . If ϕ is its local inverse,
and y = f (x), then ϕ0 (y) = f 0 (x)−1

Lemma
Shrinking Lemma : Let M be a closed subset of complete normed vector
space. Let f : M → M be a mapping, and assume there exist a number k,
0 < k < 1, such that for all x, y ∈ M we have

kf (x) − f (y)k ≤ k kx − yk

Then f has a unique fixed point, i.e. there exists a unique x0 ∈ M such that
f (x0 ) = x0 .

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Proof of Inverse Function Theorem

• Assumption : Without loss of generality, set x0 = 0, f (0) = 0, and
f 0 (0) = I.
• Let g(x) = x − f (x) such that g 0 (0) = 0. By continuity there exists an
r > 0 such that
1
g 0 (x) ≤
∀ kxk ≤ r.
2
• Continuity of f , and f (0) = 0 implies f 0 (x) is invertible for kxk ≤ r
0 0

1
• From mean value theorem we have kg(x)k ≤ 2
kxk i.e. g maps closed
ball B̄r (0) into closed ball B̄r/2 (0)
• Define gy (x) = y + x − f (x), so that it has a unique fixed point
(guaranteed by shrinking lemma) at f (x) = y.
• Let U1 = {x ∈ Br (0) : kf (x)k < r/2} and V1 = f (U1 ) be its image.
Since f : U1 → V1 is injective, inverse map exist

f : U1 → V1 f −1 = ϕ : V1 → U1 .

To show : V1 is open ϕ is of class C 1

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Proof (contd.)

• Let x1 ∈ U1 and let y1 = f (x1 ) so that ky1 k < r/2.

• For y ∈ E such that kyk < r/2 there exist a unique x ∈ B̄r (0) such
that f (x) = y. Then we have

kx − x1 k ≤ kf (x) − f (x1 )k + kg(x) − g(x1 )k

1
≤ kf (x) − f (x1 )k + kx − x1 k
2
kx − x1 k ≤ 2 kf (x) − f (x1 )k (∗)
• Hence, y is sufficiently close to y1 , if x is sufficiently close to x1 , thus
= f −1 is continuous.
• If x ∈ U1 , then y ∈ V1 and hence V1 is open.

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Proof (contd.)

• To conclude we prove differentiability of ϕ = f −1 . We know f 0 (x1 ) is

invertible because

f (x) − f (x1 ) = f 0 (x1 )(x − x1 ) + kx − x1 k ψ(x − x1 )

where limx−x1 ψ(x − x1 ) = 0.

• Substitute above result in
f −1 (y) − f −1 (y1 ) − f 0 (x1 )−1 (y − y1 ) = x − x1 − f 0 (x1 )−1 (f (x) − f (x1 )) (∗∗)
• Using (∗) and a bound C for f 0 (x1 )−1 , we obtain

k(∗∗)k ≤ 2C ky − y1 k kψ(ϕ(y) − ϕ(y1 ))k

• Continuity of ϕ = f −1 implies ϕ0 (y1 ) = f 0 (x1 )−1 . Thus, we have

ϕ0 (y) = f 0 (ϕ(y))−1

which is continuous. Thus ϕ is of class C 1 , there by completing the

proof.
• Corrollary: If f is of class C p then its local inverse is of class C p

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Outline

1 Partial Derivatives

2 Differentiablity

3 Derivatives in Vector Space

4 Inverse Function Theorem

5 Implicit Function Theorem

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Implicit Function Theorem

Theorem
Let f : U1 × U2 → R be a function of two real variables defined on a product
of two open intervals U1 , U2 . Assume that f is of class C p . Let
(a, b) ∈ U1 × U2 such that f (a, b) = 0 and D2 f (a, b) 6= 0. Then the map

ψ : U1 → U1 × U2 → R × R

given by
(x, y) 7→ (x, f (x, y))
p
is locally C invertible at (a, b)

Proof.
The Jacobian matrix of ψ at (a, b) is given by
 
1 0
Dψ(x, y) = ∂f ∂f
∂x
(a, b) ∂y (a, b)

is nonsingular at (a, b). The inverse mapping guarantees that ψ is locally

invertible at (a, b).

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Theorem
Implicit Function Theorem : Let f : U1 × U2 → R be a function of two
variables, defined on product of open interval. Let (a, b) ∈ J1 × J2 such that
f (a, b) = 0 and D2 f (a, b) 6= 0. Then there exists an open interval J ∈ R
containing a and a C p function g : J → R such that

g(a) = b and f (x, g(x) = 0 for all x ∈ J

Proof.
• ψ : U1 × U2 → R × R given by (x, y) 7→ (x, f (x, y)) is locally invertible
at (a, b).
• Let its local inverse be ϕ = ψ −1 = (ϕ1 , ϕ2 ) such that
ϕ(x, z) = (x, ϕ2 (x, z)) and let g(x) = ϕ2 (x, 0).
• ψ(a, b) = (a, 0) implies ϕ2 (a, 0) = b i.e. g(a) = b Since ψ and ϕ are
inverse mappings, we have

(x, 0) = ψ(ϕ(x, 0)) = ψ(x, g(x)) = (x, f (x, g(x)))

i.e. f (x, g(x)) = 0, proving the result.

SC 639 Spring 2020

 Partial Derivatives Differentiablity Derivatives in Vector Space Inverse Function Theorem Implicit Function Theorem

Theorem
Implicit function theorem (for Rn ): Let U be open in Rn and
f : U → R be a C p function on U . Let (a, b) = (a1 , . . . , an−1 , b) ∈ U such
that f (a, b) = 0 and Dn f (a, b) 6= 0. Then there exists an open ball V in
Rn−1 centered at a and a C p function

g:V →R

such that
g(a) = b and f (x, g(x)) = 0 for all x ∈ V

Proof.
The proof is similar to the implicit function theorem for two variables.

SC 639 Spring 2020