S. No.

Date
Title
Page
No. Teacher
Sign 's
Remarks

Deep Leaa

Mid sem 35 - 40%

ad sem ’
35-40
Tiun apr 20- 30 Yo
Read Prnt
Brnp 3-4 shuduts
Larer 020than 2020
Sowces: JMLR, TeLR, IeML,
NarIps, TPAMI, CVPR pret)
Icev. Eccv

Clas code
Loss fmehiens
Dezp Layevu/Cmponunt
Applioations
elasifitaton
DCaAN
Auoecodes WqAN

+Agmentatien 4PT

Jnatiw Models J iugle Objcchwe Toske.

’ Gbject detechion -ReNN, FRCNN, Fastin RCNN, Yo LO V2-V3
’ Fe-shot Zoro-shot task Maltiple bjctwe Tasks
 Los hmetion
Acolar Valusß
(1) 0-1 Loss
pedichßn
t: dabel
madl
qradiant ducant oill mot wnk
&calau Vaued
Evolationany Aanning olgorithms 4 Capmle met (3)
(sy gnhe alg can wok on
this pc Loss s. do mot
hag do grad. dycnt
. TToik a valid los but cawadt be wed for deup
they grad based ophinigos msdess,
() Sauaned Loss
=wx+b <-geod for nugnusion
blomieoon ?

Ni

db

Note
tet,
49 whn t=| bt
-139 whun t -I jquaned laKS.

* Mti- olas elawificatn

wSk
Dn't rk ; labels ane weed; o cene
dist on't
 tran a deep model wsg a MSE Clasifr ?
Yes.; awswes wil mot mot be geod
qrad. ule wil

* Camt trown dep

deup Mmodel
wmedel wwg
wing o- Los grad. deseanr dosnit
() bgutie Loss/Ragrsin
6(z) = l+ez

Zwtxtb

%bettu thon squared loge.

(-2I tr bim elasfction
]
= (4-t)x *e
(*e
(y-). el
(1+e
y-t)x sOx
(9-t). ez
(1+e
uppee for tol,zs -5, callote2 when d 0. 0067
Z=-l0, y =4.5x105 Strmaole tum
traming. gong
(4-y U- ) = (0.oD6 ¥-1) *o.oo63s (- s.ocr)
=-6. G|| X1o

(4.5XI0- )x 4.SX10"((- 45xI0r)

’ radient gong loo (bod)
[due to d) ] Vaihng
* Satohn roolem assoiated ith ineanay radnt problam
 "VGP trawng Atops
Sat prob ’ trawng collapsu
x Relu to tackle VGP .

() MAE LoyMean tosdwte Ernor)

model
t : labl
N N: Mo. o Aamplee m batek
* man- ditountiale at

homdle wmg Codaing eneuption

toke qradunt when mo o ; ele skip
penfect matele)

MAE nr towing
tctor-H( beten than Jquanedl wnan)
metwork. reeonotuchion or amg geatue
# As oSS ’0, trawng ons dawn n cose
case of
ike MAE . squaned berj
(5) Hubu Loss
S:thrshstd
Ls(1--s)
Squand
* MAE for te low Vales : trouing slews don for st.

tovtiwow ?
-iite ciseontinites ?
 witiaizotion -Xawer
netwvrks traumng vilile mitialization
np ás #
stop wiM tring
Prstslem' yradint trgag
toke
2add bateh,
Aample] afr *
&ealar
label t:
:model
/p
Bunany
CE ()
alasi(muti-
)
categorical crosbmoy
entry
Loss Cres (6)
Lipaclity Ls(rl-)
Mod
Lipsehit, met isLogs Hubu #
L.finite intervad,
no isthr frud Himupuwhive
hu ’
No Liptehit,?
stabilhq
Lla-| ()Il t()- || that Such
costowntL a sthoruLipsebhit,, f:R"’R"
L- tuwetin A
Lipselitg
 Tre-franed

Model 2 CModel I weight purious

trained model

Task 2
Trowatn
Leanmng
Task"
tram
the
p r e y

omodel 2

KacE -Hog y-(-t)lag(-)

z= wx+ b igned acthvstien
|+e-z

(-t
() Muti elass clastieation
(Catagrcsl ntrop)
We hawe ndieatur wer one-hot veetor
t= (0,0,..., 1,.o,o)
k-th ntny is l ift elass daal s k
K mo. elasss
(Kxi) Wx +b ’mati (Kxd)
Vetor
(ar)
Pv
Aehivotin
sftman (quunalizcd sigmnid)
Mmtivaiate 2
mtti valud]

Preve that the case

Sottman a geunalied
Itez L= wt+b
# chek

* The o/p otmoa is a taf

’ ony 1 component oill
k= Conti bute to the Loss
t’ one- hot Veetr

X
elinment-uise opunation
res ntrpy distribution a neate to distibuon ow a
given ut s definedA as

H(p.) - pl) log (

pheyricat met wnbomded
enhopy has mo mohon of Atk's like dhst b/w
oistibtHons
ions beter for elasifeotion
Oe actvay eloer
elar in the feane onse
 di - log (a) rom itth clay
K#i

k'

e e3
k=i
-

e2e .e

kzi
1
(o e

For k=L,

,Zi

+ heok

For kz i ,
Cowergne
Goung l stop vohen derivativee
’desihed (because ne- ho enerding )

dacs (o.o1, ) =- Jn (o.o1) = H.605 we mwve

Laee (0.o6 ool, 1) > -ln (o.oD oo) = ll.s13

:closz
good Coss
beter;
to actual lael

4Reqrmon Sauaned loss 2 AE Joes.

- E Loss
MMli- lalbed olaui fiotion
con hawe mtiple Aalbels
(medicad inmage elanifieatin, designing deup madel ter
pmf cEdesigndotion)
mll mot wrk
) Bnany Relevamce

Lalbel Poer det

Trect al povnble vaiants as
all those
one elas: amd fran
elaes
2 Oo ol6

K lo o 00
*froblam - No. oy elases wnereats Uponutially
9) Swm BCE
BCE k

Mode 3 k

BCEx
 + Loss wil always be sealar valued

* S3IM loss ( stuctwal nilay Tnden Meoune)

2004 TIP. Wae et. al.
3 Key termns ’ wminanee : (meau) more vaame,
Covtrst :(vanone) more contrast
2signal stuetanes :(eross eovanioucn)
Siqwal Luwminamc
measu
uminamce 3)
S 2
Comanin
Contrat
measwe
Contra
Companison Combine
Shucine
Companin
*Relounee Mehie
mud a te gromd tnth to Comþane the nesnt

# 2 2 sgnal. geuuated by metwk

romd trtn
superviud et
GAN Aettio teni atinA doun't med walowee metie ; udo labe
*global
N ’ total mo. ot tiels

Vaiance’ wmoiased estimatr ot Varianc

famçte
 Caloulate

# Lwmnamce Compariloni
+ Ci

s(xY)
N
whee

# SSIM

snal ehoiee

Man. vame of SSIM = 1 + check

" $SIM s bownded

tomdand
Vaues
(K2)

* Hiqt value
val o ssIM all eome wp whn the 2 ignals
hmilan
oun.
tuth growd ieG
Pg 2coefiint "Die
Vome
any’predcted P
þe = D
elose, how N measwus
metic grod
dirurtiabte) not opunations
ore et (:
pnloes as uied dineoty cam't
be
0. Min.
Max.
candinolity -|I0I
Union Iutrsection IoU=
Mean lace
5SIM measw boeks wto
qlabal
meoswe
isSS]M
SsIM moninize st, mn
Lo4s the into dineetiy SSIM se
camot
loss
ssM) a(1- +
uelant iwage Delhagomng *
Gombined)
ps mmboundi
las d bowrded+ "Scal(ing
SSIM 4SE
Mac
G[mam
aligrud)
M. vae
[min ownlap
(" D hoas to be manimiged so
Less can't have D direety)

-2 z8)-g(24)
(*)
* Tvesky Loss
Salei et al. IWML, 2017
Haward Medical cho

+Tonay Celficient / Dndua

|xn|
s(x1) =

spcial Case

T(B) N

L=1
’3-D voma
o Vozel i to be deion
wNon lesion (omplemnt Po)
for unin -volume
Lo 6uise
l for won-lewion volume
 Mox valuu of T dapende onehaiee o t f =|

* Chaen Ditane
"ehoee he manet
GT towt to
caleilat lee.

wim
der (s, s) - Xes,Xgesp -4 n

tobe
mùnimized X’ pred drppgthe.ud tom wl
Y ’6T ost

* KL Diwgune

9()
KL- Div. rP
wrt
+ nelatin bw
- log s(). KL div, L
CE

=Hlp.4) - H)
 Rroe r dus prove that D(Pl 4) o
Sanaun's Tnaqality
fumchon,

a(t())
P(*)
to mate t
Cowe P(E0)
q(E)
P(E())
Zr) Qa)
X PC9
(E)
- las 1. E)) E()=1
logP)
g().
Dup Comtonanta
- Fllhy Cowectad laues (Fc) [NN pont
[NN pant nn PRMI
-Activations/ nNan- ininitg
- Conwodtion Layes (Flatening) " Tensor
- Poo ling lager mti dimensibnal
motrix.
Racwnent Layo
Residuat ayo
- lerti¡n Xager

Back prep
Dwivaiale/wuti vaniate nchions ith scala ovtpt (hwngle
valued fumchii
Can be Scalarvector
0 Lieor Aetivotion
a) e.x
Wse

ith tve -e vaes

Sigmod Achivation
)= |+e

or binany clawifcation
tanh funchion
() = e voishing grndirt
prdolem
Lstsse

|+e-2a

() (+e)(2) - (I-e")2e)
(te
2e- (1 te +|-e)
(ite)
4e-2a

(1+ey
’ ( ) = |- 44)
Max. valwe oy (2) = |
Min. vale ot () -I
ARuchfied Linan Dnit (Ralu) to miti¡ate the vomishing
gradibt prtdam
detoult choie for hidden layer acivotien
)-maa (0, )
* hoween, mot differewrtiable

. ut side s L- Lipschi
" Hve hde
voihing gradint doroblem
-e yide stil has avamihing grndiint protlem
* Relu wvdues Aponialy to the metwrk
s(w*+)
wXtb ’-Ve. RelU ill Ave o

"praing
* Dyng Relu problerm furthen, oconbibution
it mihoizotion is bad, if you hawe wìll mo ^tßeet
nate
igh
owtx+ b)
-ve bios
l node
nod will go to at vey eany stage of

Laky RelU
tra~iing
('wnat hronabl)
a : 2 for ao akage barauatie

typieally, walue t a is vey small

"L- Lipsitg
wn-dylnertiole ’ codingeception
6 Panametie RelU (PRelu)) rodes but uult
hawere, no. o
for o trauinatte panametou
da for x<o; ao
tainable

Bottphs Activation

Swooth approx? Relu

ReLw for hiah vaue of B
220’ l+e
ac0’l+eIeo
8) Erpontal linean Unit (Eu)

L’ Conctimows tor a=|

1

mn. Vyalue =-1

Condimonsy itnentinde ELU (CELU

graduint in smal,
enplosion vau mot happen