The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Learning to Adapt Neural Networks Across Visual Domains

Ph.D. Defense

Ph.D. Candidate: Subhankar Roy

Advisors: Prof. Elisa Ricci and Prof. Nicu Sebe

Università degli studi di Trento

September 29, 2022

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 1 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overview

1 The Need to Adapt Neural Networks

2 Unsupervised Domain Adaptation

3 Contributions
Domain Whitening Transform (CVPR’ 19)
TriGAN for Multi-source Domain Adaptation (MVA’ 21)
Curriculum Graph Co-teaching for Multi-target Domain Adaptation (CVPR’ 21)
Uncertainty-aware Source-free Domain Adaptation (ECCV’ 22)

4 Conclusions

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 2 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overview

1 The Need to Adapt Neural Networks

2 Unsupervised Domain Adaptation

3 Contributions

4 Conclusions

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 3 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

The Need to Adapt Neural Networks

▶ In deep learning we aim to learn an

approximating function fθ : X → Y Train = Test
with a deep neural network, given
some data (X , Y ).
▶ Typically, it is assumed that the MNIST MNIST
training and testing data comes from
the same underlying distribution (or
i.i.d). SVHN SVHN
▶ Neural networks are fragile when the Independent and Identically Distributed (i.i.d)
i.i.d assumption is violated.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 4 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain-Shift Problem

Train (or Source) ̸= Test (or Target)

▶ A source-trained model when tested
on a different test distribution (a.k.a
domain ) yields sub-optimal
MNIST SVHN
performance.
▶ The shift in distributions (or visual
change in appearance between
domains) is also known as the SVHN MNIST
domain-shift problem (Torralba and
Efros, 2011).
Amazon Products Real World

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 5 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Generalization Gap

Train Test Accuracy

99.3
20.2 (↓79.7%)

96.7
60.1 (↓37.8%)
Image courtesy: PhotobookUK

Table 1: The generalization gap caused by the ▶ Collecting labelled data from
domain-shift every test domain is prohibitive.
▶ Can we do better?

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 6 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overview

1 The Need to Adapt Neural Networks

2 Unsupervised Domain Adaptation

3 Contributions

4 Conclusions

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 7 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Unsupervised Domain Adaptation

▶ Collecting unlabelled data from the target domain is easy and cheap .
▶ Given an labelled dataset (DS ) from source domain (S) and an unlabelled
dataset (DT ) from the target domain (T ), Unsupervised Domain Adaptation
(UDA) (Ben-David et al., 2010; Csurka, 2017) deals with adapting the model to
work well in the target domain.
DS DT DT
Train

Test
Bed Cycle Kettle ?
▶ Assumptions:
pS (y|x) = pT (y|x)

pS (x) ̸= pT (x)
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 8 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Facets of Unsupervised Domain Adaptation

Figure 1: UDA can be addressed under different settings depending upon the assumptions
made on the source and target domains. Each setting has its own challenges.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 9 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Applications

Figure 2: Example scenarios where domain-shift can manifest in real-world applications. UDA
techniques can help tackle such challenges

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 10 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Benchmarks
Digits (5 domains) Visda-C (2 domains) Domain Net (6 domains)

MNIST
Synthetic Images Painting Infograph

USPS

Sketch Clipart
Color-MNIST Real World objects from COCO
CIFAR-STL (2 domains)

Real World Quick Draw

SVHN
CIFAR10 Office-31 (3 domains)

Synthetic Digits
STL10

Office-Home (4 domains) Office-Caltech (4 domains)

DSLR

Art Product Amazon DSLR Webcam

Clipart Real World Webcam Caltech Amazon

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 11 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overview

1 The Need to Adapt Neural Networks

2 Unsupervised Domain Adaptation

3 Contributions
Domain Whitening Transform (CVPR’ 19)
TriGAN for Multi-source Domain Adaptation (MVA’ 21)
Curriculum Graph Co-teaching for Multi-target Domain Adaptation (CVPR’ 21)
Uncertainty-aware Source-free Domain Adaptation (ECCV’ 22)

4 Conclusions

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 12 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Contributions

The contributions in this doctoral study are all aimed at the following research avenues:
▶ To understand and discover the tenets of overcoming domain-shift in image
classification tasks, starting from single-source single-target DA (STDA) setting.
▶ To improve upon the existing state-of-the-art STDA methods.
▶ To progressively address more practical and real-world DA scenarios.
▶ To explore and bring interesting ideas from the field of machine learning to the
computer vision community, in the context of neural network adaptation.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 13 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain Whitening Transform (CVPR’ 19)

The first contribution falls under the family of feature-alignment UDA methods.
▶ Use Batch Normalization-based
embedded Source
features
Domain Alignment layers (Carlucci
Standardization
et al., 2017) for aligning marginal Target
features
feature distributions between the
source and target.
▶ Domain Alignment layers align feature
Standardization
distributions through feature
standardization by setting mean of
features to 0 and variance to 1.
▶ It leaves the feature correlations
Figure 3: BN-based Domain Alignment
intact, leading to imperfect alignment.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 14 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain Whitening Transform

▶ We proposed domain-specific Domain
Source
Whitening Transform ( DWT ) to features

project the intermediate features for Whitening

Target
each domain independently, such that features

their covariance matrices are identical.

▶ DWT not only projects the
intermediate features to a distribution
Whitening
with zero mean but also decorrelates
the features.
▶ Projection to the canonical
representation allows to align the two
Figure 4: Whitening-based Domain Alignment
domains.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 15 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain-specific Whitening Transform

▶ We propose to use domain-specific Batch Whitening (BW) to align domains by
projecting the features to a common hyperspherical feature distribution which is
defined as:
BW(xi,k ; Ω) = γk x̂i,k + βk , (1)
x̂i = WB (xi − µB ). (2)

where

WB⊤ WB = Σ−1
B , and Ω = (µB , ΣB )
are domain-specific and are domain-agnostic.
▶ WB is the Whitening matrix. (µB , ΣB ) are domain-specific mean and covariance.
γ and β are the learnable parameters as in BN (Ioffe and Szegedy, 2015).

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 16 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain-specific Whitening Transform

(a) (b) (c)

Figure 5: Covariance matrices of features undergoing different normalization transformations:
(a) BN (Carlucci et al., 2017); (b) grouped-Whitening; and (c) Full-Feature Whitening. Black
pixels denote value 1, white pixels denote value 0 and gray denotes intermediate values

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 17 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Minimum Entropy Consensus Loss

To better utilize the unlabelled target data, we propose the Min-Entropy Consensus Loss
that optimizes the following simultaneously:
▶ Minimizes the entropy to ensure that the predictor maximally separates the
target data
▶ Minimizes the consistency loss to force the predictor to deliver consistent
predictions for target samples
MEC Loss is given as:
1  
ℓt ( xti 1 , xti 2 ) = − arg max log p(y | xti 1 ) + log p(y | xti 2 ) . (3)
2 y ∈Y

where xti 1 and xti 2 are augmented versions of the same target sample xit

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 18 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overall Pipeline

Figure 6: Overview of the proposed deep architecture containing the embedded DWT layers,
which is trained with the Cross Entropy and MEC loss
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 19 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Digits
MNIST USPS SVHN MNIST
Methods USPS MNIST MNIST SVHN
Source Only 78.9 57.1±1.7 60.1±1.1 20.23±1.8
CORAL (Sun et al., 2016) 81.7 - 63.1 -
MMD (Tzeng et al., 2015) 81.1 - 71.1 -
DANN (Ganin et al., 2016) 85.1 73.0±2.0 73.9 35.7
w/o aug

ADDA (Tzeng et al., 2017) 89.4±0.2 90.1±0.8 76.0±1.8 -

DRCN (Ghifary et al., 2016) 91.8±0.1 73.7±0.1 82.0±0.2 40.1±0.1
AutoDIAL (Carlucci et al., 2017) 97.96 97.51 89.12 10.78
SBADA-GAN (Russo et al., 2018) 97.6 95.0 76.1 61.1
GAM (Huang et al., 2018) 95.7±0.5 98.0±0.5 74.6±1.1 -
DWT 99.09±0.09 98.79±0.05 97.75±0.10 28.92 ±1.9
SE a (French et al., 2018) 88.14±0.34 92.35±8.61 93.33±5.88 33.87±4.02
w/ aug

SE b (French et al., 2018) 98.23±0.13 99.54±0.04 99.26±0.05 37.49±2.44

DWT-MECb 99.01±0.06 99.02±0.05 97.80±0.07 30.20±0.92
DWT-MEC (MT)b 99.30±0.19 99.15±0.05 99.14±0.02 31.58±2.34
Table 2: Comparison with the state-of-the-art on Digits. a indicates minimal usage of data
augmentation and b considers augmented source and target data
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 20 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on CIFAR-STL
CIFAR-10 STL
Methods STL CIFAR-10
Source Only 60.35 51.88
w/o aug DANN (Ganin et al., 2016) 66.12 56.91
DRCN (Ghifary et al., 2016) 66.37 58.65
AutoDIAL (Carlucci et al., 2017) 79.10 70.15
DWT 79.75±0.25 71.18±0.56
SE a (French et al., 2018) 77.53±0.11 71.65±0.67
w/ aug

SE b (French et al., 2018) 80.09±0.31 69.86±1.97

DWT-MECb 80.39±0.31 72.52±0.94
DWT-MEC (MT)b 81.83±0.14 71.31±0.22
a
Table 3: Comparison with the state-of-the-art on the CIFAR-10↔STL. indicates minimal data
augmentation and b considers augmented source and target data.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 21 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Office-Home

Ar Ar Ar Cl Cl Cl Pr Pr Pr Rw Rw Rw
Methods Cl Pr Rw Ar Pr Rw Ar Cl Rw Ar Cl Pr Avg
Source Only 34.9 50.0 58.0 37.4 41.9 46.2 38.5 31.2 60.4 53.9 41.2 59.9 46.1
DAN (Long and Wang, 2015) 43.6 57.0 67.9 45.8 56.5 60.4 44.0 43.6 67.7 63.1 51.5 74.3 56.3
DANN (Ganin et al., 2016) 45.6 59.3 70.1 47.0 58.5 60.9 46.1 43.7 68.5 63.2 51.8 76.8 57.6
JAN (Long et al., 2017) 45.9 61.2 68.9 50.4 59.7 61.0 45.8 43.4 70.3 63.9 52.4 76.8 58.3
SE (French et al., 2018) 48.8 61.8 72.8 54.1 63.2 65.1 50.6 49.2 72.3 66.1 55.9 78.7 61.5
CDAN-RM (Long et al., 2018) 49.2 64.8 72.9 53.8 63.9 62.9 49.8 48.8 71.5 65.8 56.4 79.2 61.6
CDAN-M (Long et al., 2018) 50.6 65.9 73.4 55.7 62.7 64.2 51.8 49.1 74.5 68.2 56.9 80.7 62.8
DWT-MEC 50.3 72.1 77.0 59.6 69.3 70.2 58.3 48.1 77.3 69.3 53.6 82.0 65.6

Table 4: Comparison with the state-of-the-art methods on Office-Home dataset with ResNet-50
as encoder. Art (Ar), Clipart (Cl), Product (Pr) and Real-World (Rw) are the four domains in
the Office-Home dataset.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 22 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

TriGAN for Multi-source Domain Adaptation (MVA’ 21)

The second contribution falls under the family of generative UDA methods where the
key idea is to use CycleGAN (Zhu et al., 2017) to learn unpaired translation in the pixel
space, between a pair of domains.

(xS , ‘bag’) CycleGAN (xS→T , ‘bag’)

Source CycleGAN Target

bag/no bag
Figure 7: Unpaired image translation

(xS→T , ‘bag’) Classifier

Figure 8: Train target classifier on generated images
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 23 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

TriGAN for Multi-source Domain Adaptation

Figure 9: Pipeline for unpaired image translation between a pair of domains (autumn and
summer)
Challenges in Multi-source DA with a generative framework:
▶ Number of generators and discriminators scale linearly with number of source
domains.
▶ Can not utilize all the data from different source domains.
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 24 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

TriGAN for Multi-source Domain Adaptation

In multi-source DA (MSDA) the task is to adapt from multiple labelled source domains
to one unlabelled target domain. In this work we propose two main contributions:
▶ An universal generator that translates images from multiple source domains into
the target domain.
■ Reduces redundancy in network weights, allowing weights sharing.
■ The generator can jointly use data from all the source domains.
▶ We design our generator that assumes the appearance of an image depends on
three factors:
■ the semantic content
■ the domain that models properties shared uniquely by samples of a dataset
■ the style or low-level features unique to each sample
MNIST SVHN Synthetic Digits

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 25 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

TriGAN Universal Generator

Encoder Decoder
T Style
Path

S1

S2 IWT DWT cDWT Adaptive

blocks blocks blocks IWT
blocks

Style invariant Domain invariant Domain specific

representation representation representation

Figure 10: The content is represented by the object’s shape - square, circle or triangle. The
domain is represented by holes, 3D objects and skewered. The style is represented by the color.
▶ The encoder sequentially removes style and domain information to obtain
domain invariant representation.
▶ The decoder symmetrically projects them to domain-specific and style-specific
distribution.
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 26 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Whitening and Coloring Transform (WC)

We re-use the domain-specific whitening transform from the previous work and replace
scale-shift parameters (β, γ) with the coloring transform (Li et al., 2017; Siarohin et al.,
2019).
m
▶ Given, Xi = {v1 , ..., vh×w } and B = ∪ Xi = {v1 , ..., vh×w ×m } we define WC as:
i=1

WC (vj ; B, β, Γ) = Coloring (v̄j ; β, Γ) = Γ v̄j + β (4)

where:
−1
v̄j = Whitening (vj ; B) = W B (vj − µB ), s.t. W ⊤
B W B = ΣB (5)

are domain-specific and are domain-agnostic.

▶ β and Γ are the d and d × d dimensional coloring parameters, respectively, which
are analogous to the BN (Ioffe and Szegedy, 2015) learnable parameters.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 27 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Style Invariant Representation

▶ To remove the style from an image,

inspired by Instance Normalization
(IN) (Ulyanov et al., 2016), we use
Instance Whitening Transform (IWT)
blocks:

IWT (vj ) = WC (vj ; Xi , β, Γ). (6)

▶ We apply IWT in the first two

Figure 11: Instance Whitening Transform convolutional blocks of the encoder.
(IWT) to obtain style invariant representation

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 28 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain Invariant Representation

▶ Given a sub-set of a batch of images

Bli ⊆ B from a specific domain li , we
perform DWT (from our previous
work (Roy et al., 2019)) to obtain
domain invariant representation as:

DWT (vj ) = WC (vj ; Bli , β, Γ). (7)

Figure 12: Domain Whitening Transform
(DWT) to obtain domain invariant
representation

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 29 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain-specific Representation

▶ In the decoder, we project the domain

invariant representation to
domain-specific representation by
conditioning on a desired domain label
liO .
▶ The cDWT block is given as:

cDWT (vj ) = WC (vj ; Bl O , β l O , Γl O ).

i i i
(8)
▶ The parameters (β l , Γl ) are
Figure 13: conditional Domain Whitening
Transform (cDWT) to obtain domain-specific conditioned on liO .
representation

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 30 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Re-projecting to Output Pixel Space

▶ The Adaptive Instance Whitening

Transform (AdaIWT) applies the style
from a reference image xOi to
re-project domain-specific
representation to the pixel space:
AdaIWT (vj ) = WC (vj ; XiO , β i , Γi ).
(9)
▶ The coloring parameters in AdaIWT
(β i , Γi ) is obtained using the Style
Path as:
Figure 14: Adaptive Instance Whitening
Transform (AdaIWT) to re-project to output [β i ∥Γi ] = F([µX O ∥W −1
XO
]). (10)
i i
pixel space.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 31 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

TriGAN Training Objective

Let x̂i = G (xi , li , liO , xO
i ) be the generated image from the generator (G), where xi is
input image from domain li , liO is the output domain label and xO i is the desired style
image.
We train TriGAN using the following objectives:
▶ A standard adversarial loss to train the generator and the discriminator (D).
▶ Identity loss LID (G ) = ||G (xi , li , li , xi ) − xi ||1 to train G.
▶ Equaviarance loss LEq (G ) = ||G (h(xi ; θ i ), li , liO , xO
i ) − h(x̂i ; θ i )||1 to train G in
order to ensure geometric equivariance, where {h(x; θ)} are simple 2D affine
transformations.
▶ Finally, a standard cross-entropy loss to train target-specific classifier on
generated target images.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 32 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Digits-Five

mt, up, sv, sy mm, up, sv, sy mt, mm, sv, sy mt, up, mm, sy mt, up, sv, mm
Settings Methods Avg
→ mm → mt → up → sv → sy
Source Only 63.70±0.83 92.30±0.91 90.71±0.54 71.51±0.75 83.44±0.79 80.33
Source
DAN (Long and Wang, 2015) 67.87±0.75 97.50±0.62 93.49±0.85 67.80±0.84 86.93±0.93 82.72
Combine
DANN (Ganin et al., 2016) 70.81±0.94 97.90±0.83 93.47±0.79 68.50±0.85 87.37±0.68 83.61
Source Only 63.37±0.74 90.50±0.83 88.71±0.89 63.54±0.93 82.44±0.65 77.71
DAN (Long and Wang, 2015) 63.78±0.71 96.31±0.54 94.24±0.87 62.45±0.72 85.43±0.77 80.44
CORAL (Sun et al., 2016) 62.53±0.69 97.21±0.83 93.45±0.82 64.40±0.72 82.77±0.69 80.07
Multi- DANN (Ganin et al., 2016) 71.30±0.56 97.60±0.75 92.33±0.85 63.48±0.79 85.34±0.84 82.01
Source ADDA (Tzeng et al., 2017) 71.57±0.52 97.89±0.84 92.83±0.74 75.48±0.48 86.45±0.62 84.84
DCTN (Xu et al., 2018) 70.53±1.24 96.23±0.82 92.81±0.27 77.61±0.41 86.77±0.78 84.79
M3 SDA (Peng et al., 2019a) 72.82±1.13 98.43±0.68 96.14±0.81 81.32±0.86 89.58±0.56 87.65
StarGAN (Choi et al., 2018) 44.71±1.39 96.26±0.62 55.32±3.71 58.93±1.95 63.36±2.41 63.71
TriGAN 83.20±0.78 97.20±0.45 94.08±0.92 85.66±0.79 90.30±0.57 90.08

Table 5: Comparison with the multi-source DA state-of-the-art on Digits-Five. MNIST-M,

MNIST, USPS, SVHN, Synthetic Digits are abbreviated as mm, mt, up, sv and sy
respectively. Best number is in bold and second best is underlined

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 33 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Office-Caltech

Standards Methods A,D,C→W A,W,C→D A,W,D→C W,D,C→A Avg

Source Source only 99.0 98.3 87.8 86.1 92.8
Combine DAN (Long and Wang, 2015) 99.3 98.2 89.7 94.8 95.5
Source only 99.1 98.2 85.4 88.7 92.9
DAN (Long and Wang, 2015) 99.5 99.1 89.2 91.6 94.8
Multi-
DCTN (Xu et al., 2018) 99.4 99.0 90.2 92.7 95.3
Source
M3 SDA (Peng et al., 2019a) 99.5 99.2 92.2 94.5 96.4
StarGAN (Choi et al., 2018) 99.6 100.0 89.3 93.3 95.5
TriGAN 99.7 100.0 93.0 95.2 97.0

Table 6: Comparison with the multi-source DA state-of-the-art on the Office-Caltech10.

Amazon (A), Caltech (C), DSLR (D) and Webcam (W) are the four domains in the
Office-Caltech10 dataset

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 34 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Ablation Analysis of TriGAN

Avg. Accuracy (%)
Model Description
(Abs. Difference)
A TriGAN (full method) 90.08
Replace Equivariance Loss
B 88.38 (-1.70)
with Cycle Loss
Replace Whitening with
C 89.39 (-0.68)
Feature Standardisation
D No Style Assumption 88.32 (-1.76)
Applying style directly
E 88.36 (-1.71)
instead of style path
F No Domain Assumption 89.10 (-0.98)
No Style Assumption,
StarGAN Domain Labels concatenated 63.71 (-26.37)
with the input image

Table 7: Ablation analysis of the main TriGAN components using Digits-Five dataset

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 35 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Single Source DA

MNIST MNIST MNIST
Methods
USPS MNIST-M SVHN
Source Only 78.9 63.6 26.0
DANN (Ganin et al., 2016) 85.1 77.4 35.7
CoGAN (Liu and Tuzel, 2016) 91.2 62.0 -
ADDA (Tzeng et al., 2017) 89.4 - -
PixelDA (Bousmalis et al., 2017) 95.9 98.2 -
UNIT (Liu et al., 2017) 95.9 - -
SBADA-GAN (Russo et al., 2018) 97.6 99.4 61.1
GenToAdapt (Sankaranarayanan et al., 2018) 92.5 - 36.4
CyCADA (Hoffman et al., 2017) 94.8 - -
I2I Adapt (Murez et al., 2018) 92.1 - -
TriGAN 98.0 95.7 66.3

Table 8: Comparison with the state-of-the-art GAN-based methods in the Single-source UDA
setting on Digits. The best number is in bold and the second best is underlined

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 36 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Digits-Five Domain Translation

MNIST MNIST-M SVHN SYNDIGITS USPS

St rc
S
ou
yl e
e
MNIST

MNIST-M

SVHN

SYNDIGITS

USPS

Figure 15: Generations of TriGAN across different domains of Digits-Five. Leftmost column
shows the source images, one from each domain and the topmost row shows the style image
from the target domain, two from each domain

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 37 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Office-Caltech Domain Translation

Caltech DSLR Webcam Amazon

St rc
S
ou
yl e
e
Caltech

DSLR

Webcam

Amazon

Figure 16: Generations of TriGAN across different domains of Office-Caltech. Leftmost column
shows the source images, one from each domain and the topmost row shows the style image
from the target domain, two from each domain

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 38 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Alps Seasons Domain Translation

So

S
u

ty
rc

le
e

Spring

Summer

Winter

Autumn

Figure 17: Generations of TriGAN across different domains of Alps Seasons. Leftmost column
shows the source images, one from each domain and the topmost row shows the style image
from the target domain, two from each domain

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 39 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Curriculum Graph Co-teaching for Multi-target Domain Adaptation

(CVPR’ 21)
In this work we tackle multi-target DA (MTDA) where the task is to adapt a single model
from a single labelled source domain to multiple unlabelled target domains.

Figure 18: Schematic representation of multiple Figure 19: An instance of real-world application
domain-shifts w.r.t.the target in MTDA where MTDA is of significance

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 40 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Multi-target Domain Adaptation

Why is the problem relevant?

▶ Real world is more complex and contains more
than just a single target domain.
▶ Number of networks scales linearly with the
number of target domains.
▶ No domain id available during inference.

Why is it challenging?
▶ Multiple and varying domain shifts.
▶ Non-overlapping support with the source.
▶ Problem of negative transfer .

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 41 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Curriculum Graph Co-teaching for Multi-target Domain Adaptation

In this work we propose the following contributions to address MTDA:

▶ Feature aggregation with Graph Convolutional Neural Network (GCN) in order
to obtain domain invariant representation.
▶ Co-teaching with a dual-head classifier for more accurate pseudo-labelling.
▶ Domain curriculum learning for a smoother adaptation across multiple domains.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 42 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Feature Aggregation with GCN

They key idea is to use a graph neural network to aggregate the features of similar samples
across all the domains. This results in the alignment of feature distributions.
▶ We assume an undirected and fully-connected
graph Γ = (F, E, A) for a mini-batch B.
▶ For nodes f ∈ F, αi,j ∈ A is the learnt
semantic similarity between fi and fj , given as:
(l) (l) (l−1) (l−1)
αi,j = Fedge (fi , fj ), (11)

▶ The aggregated features of node fi in layer l is:

 X (l) (l−1) 
(l) (l) (l−1)
fi = Fnode [fi , αi,j · fj ] , (12)
j∈B
Figure 20: Schematic feature
aggregation in a node of a GCN ▶ Training Fedge and Fnode requires class labels!

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 43 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Co-teaching with Dual-head Classifier

Figure 21: To prevent error propagation in the GCN, we adopt co-teaching with a dual-head
classifier. It results in improved pseudo-labels for the target samples
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 44 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overall Pipeline

Figure 22: Overall framework of curriculum graph co-teaching (CGCT) for multi-target DA

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 45 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Training Objectives

Training the overall framework includes training the shared feature extractor, domain
discriminator, the MLP classifier Gmlp , and the GCN classifier Ggcn , which is composed
of edge network Fedge and node classifier Fnode . For the target samples:
▶ Fedge is trained with binary cross-entropy loss using pseudo-labels from Gmlp .
▶ Fnode is trained with cross-entropy loss using pseudo-labels from Gmlp .
▶ Gmlp is trained with cross-entropy loss using the pseudo-labels obtained from Ggcn
▶ We also use the adversarial loss in CDAN (Long et al., 2018) to ensure even
better domain alignment.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 46 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Domain Curriculum Learning

Figure 23: Domain Curriculum Learning is based on Easy-to-Hard Domain Selection (EHDS)
strategy. Adaptation process starts with the easiest domain and proceeds to the hardest one.
Expected entropy is used as a metric to sort the domains
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 47 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Digits-Five

Digits-five
Settings Methods MNIST MNIST-M SVHN Syn. Digits USPS Avg (%)
Source Only 26.9 56.0 67.2 73.8 36.9 52.2
ADDA (Tzeng et al., 2017) 43.7 55.9 40.4 66.1 34.8 48.2
DAN (Long and Wang, 2015) 31.3 53.1 48.7 63.3 27.0 44.7
Target GTA (Sankaranarayanan et al., 2018) 44.6 54.5 60.3 74.5 41.3 55.0
Combined DANN (Ganin et al., 2016) 52.4 64.0 65.3 66.6 44.3 58.5
AMEAN (Chen et al., 2019b) 56.2 65.2 67.3 71.3 47.5 61.5
CDAN (Long et al., 2018) 53.0 76.3 65.6 81.5 56.2 66.5
CGCT 54.3 85.5 83.8 87.8 52.4 72.8
CDAN (Long et al., 2018) 53.7 76.2 64.4 80.3 46.2 64.2
Multi-
CDAN + DCL 62.0 87.8 87.8 92.3 63.2 78.6
Target
D-CGCT 65.7 89.0 88.9 93.2 62.9 79.9

Table 9: Comparison with the MTDA state-of-the-art methods on the Digits-five

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 48 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Office-31 and Office-Home

Office-31 Office-Home
Settings Methods Amazon DSLR Webcam Avg(%) Art Clipart Product Real Avg(%)
w/o Target Source Only 68.6 70.0 66.5 68.4 47.6 42.6 44.2 51.3 46.4
DAN (Long and Wang, 2015) 79.5 80.3 81.2 80.4 56.1 54.2 51.7 63.0 56.3
DANN (Ganin et al., 2016) 80.8 82.5 83.2 82.2 58.3 55.4 52.8 63.9 57.6
Single-
JAN (Long et al., 2017) 85.0 83.0 85.6 84.3 58.7 57.0 53.1 64.3 58.3
Target
CDAN (Long et al., 2018) 91.4 84.1 84.0 86.6 64.2 62.9 59.9 68.1 63.8
CGCT 89.6 85.5 87.6 87.6 67.9 68.7 62.3 70.7 67.4
DAN (Long and Wang, 2015) 78.0 64.4 66.7 69.7 55.6 56.6 48.5 56.7 54.4
DANN (Ganin et al., 2016) 78.2 72.2 69.8 73.4 58.4 58.1 52.9 62.1 57.9
Target- JAN (Long et al., 2017) 84.2 74.4 72.0 76.9 58.3 60.5 52.2 57.5 57.1
Combined CDAN (Long et al., 2018) 93.6 80.5 81.3 85.1 59.5 61.0 54.7 62.9 59.5
AMEAN (Chen et al., 2019b) 90.1 77.0 73.4 80.2 64.3 65.5 59.5 66.7 64.0
CGCT 93.9 85.1 85.6 88.2 67.4 68.1 61.6 68.7 66.5
MT-MTDA (Nguyen et al., 2021) 87.9 83.7 84.0 85.2 64.6 66.4 59.2 67.1 64.3
Multi- HGAN (Yang et al., 2020) 88.0 84.4 84.9 85.8 - - - - -
Target CDAN+DCL 92.6 82.5 84.7 86.6 63.0 66.3 60.0 67.0 64.1
D-CGCT 93.4 86.0 87.1 88.8 70.5 71.6 66.0 71.2 69.8

Table 10: Comparison with the MTDA state-of-the-art methods on Office-31 and Office-Home

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 49 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Domain Net

DomainNet
Methods Clip Info Paint Quick Draw Real Sketch Avg(%)
Source Only 25.6 16.8 25.8 9.2 20.6 22.3 20.1
SE (French et al., 2018) 21.3 8.5 14.5 13.8 16.0 19.7 15.6
MCD (Saito et al., 2018) 25.1 19.1 27.0 10.4 20.2 22.5 20.7
DADA (Peng et al., 2019b) 26.1 20.0 26.5 12.9 20.7 22.8 21.5
CDAN (Long et al., 2018) 31.6 27.1 31.8 12.5 33.2 35.8 28.7
MCC (Jin et al., 2020) 33.6 30.0 32.4 13.5 28.0 35.3 28.8
CDAN + DCL 35.1 31.4 37.0 20.5 35.4 41.0 33.4
CGCT 36.1 33.3 35.0 10.0 39.6 39.7 32.3
D-CGCT 37.0 32.2 37.3 19.3 39.8 40.8 34.4

Table 11: Comparison with the MTDA state-of-the-art methods on DomainNet. All the
reported numbers are evaluated on the multi-target setting.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 50 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Ablation Analysis
Office-Home
Methods Art Clipart Product Real Avg(%)
Source train 51.45 43.93 42.41 54.50 48.07
CDAN (Baseline) 50.70 50.78 47.95 57.63 51.77
Baseline† 52.08 53.21 48.62 58.49 53.10
Baseline† +PL 54.61 56.13 50.25 61.04 55.51
Baseline† + DCL 55.94 56.66 52.85 60.18 56.41
Baseline† +GCN‡ 50.19 49.09 46.52 60.76 51.64
Baseline† +GCN‡ + PL 54.52 57.60 53.20 65.49 57.70
CGCT 60.81 60.00 54.13 62.62 59.39
D-CGCT 61.42 60.73 57.27 63.8 60.81

Table 12: Ablation analysis on Office-Home. Baseline: CDAN model that combines all the
target domains. “†”: the baseline models that use the domain labels of the target. GCN‡: the
baseline model with the GCN as the single classification head. PL: using pseudo-labels.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 51 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Ablation Analysis
Office-Home Digits-five
70 100
Reverse-domain Curriculum
Domain-aware Curriculum
Accuracy (%)

60 85

Accuracy (%)
50 70

40 55

30 40
Art Clipart Product Real mt mm svhn syn usps
Source Domain Source Domain
Figure 24: Comparison of the DCL with the reverse-domain curriculum model on Office-Home
and Digits-Five. In the reverse-domain curriculum model the order of selection of target
domains is exactly opposite to that of the DCL model.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 52 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Uncertainty-aware Source-free Domain Adaptation (ECCV’ 22)

▶ In the works seen so far, both the source and target datasets are needed while
carrying out the model adaptation.
▶ Rising data privacy regulations and storage issues are bottlenecks!
▶ Source-free domain adaptation (SFDA) aims to adapt a classifier to an
unlabelled target dataset by only using a pre-trained source model.

Figure 25: Illustration of source-free domain adaptation scenario

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 53 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Source-Free Domain Adaptation

Source Hypothesis Transfer (SHOT) (Liang et al., 2020) is a popular and effective
approach to address SFDA.

Figure 27: Effect of Information Maximization

K
X
Lent =−Ep(x[T] ) gt (hs (xk [T] )) log gt (hs (xk [T] )),
Figure 26: In SHOT feature extractor is k=1
updated whereas the classifier (hypotheis) is (13)
kept frozen during adaptation −1
Ldiv =DKL (p̂ ∥ K 1K ) − log K , (14)

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 54 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overconfident Predictions
Hein et al. (2019) showed that ReLU networks due to the extrapolation property tend
to produce high confident predictions far from training data points.

▶ ReLU networks do not come with a

notion of uncertainty out of the box.
▶ Under strong domain-shift target data
can lie far from from source
distribution.
▶ As source data is absent , optimizing
IM loss (Liang et al., 2020) can lead
to erroneous decision surfaces.
Figure 28: The data points ( ) of an unseen
class is classified with high confidence

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 55 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Bayesian Deep Learning

Bayesian deep learning (Neal, 2012) is concerned with formalizing prior knowledge and
representing uncertainty in model predictions, especially under domain-shift or out-of-
distribution samples.

▶ Bayesian neural networks “know”

when they do not know.
▶ Due to computational intractibility
approximate inference methods such
as MC dropout (Gal and
Ghahramani, 2016),
deep ensembles (Lakshminarayanan
et al., 2017),
variational inference (Blei et al.,
Figure 29: Predictive uncertainty captured by a 2017), etc are adopted.
Bayesian neural network

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 56 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Uncertainty-aware Source-free Adaptation

In this work we propose to adopt a probabilistic model in order incorporate the uncer-
tainties about the model predictions on the target data into the adaptation process.

▶ Issues of popular Bayesian approaches:

■ MC dropout requires re-training

with dropout layers, configuration
of dropout layer, etc.
■ Deep ensembles need computations
and careful initialization . Figure 30: Laplace approximation. Weights are
▶ Need of strict de-coupling of source linearized around the point estimate θMAP
and target training. ▶ Impose a Gaussian structure over the
network weights.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 57 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Laplace Approximation
▶ Instead of a point estimate θ, we want a posterior distribution over the model
p(θ) p(D|θ)
parameters p(θ | D) = p(D)
▶ Prediction requires marginalization over the weights:

Z
p(yk | x, D) = ϕk (fθ (x)) p(θ | D) dθ. (15)
θ

where ϕ(·) is the softmax operator and fθ is the neural network.

▶ The posterior p(θ | D) in Eq. (15) does not have an analytical solution and need
to be approximated.
▶ A local approximation to the true posterior is made using Laplace Approximation
(LA) (Tierney and Kadane, 1986).

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 58 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Laplace Approximation
▶ A trained neural network gives the local maximum as θMAP .
▶ LA uses the second-order Taylor expansion around θMAP to give an approximated
multivariate Gaussian distribution as:

p(θ | D) ≈ N(θ | θMAP , H−1 ) (16)

where the covariance matrix is given by inverse of the Hessian H of the negative
log-posterior H := −∇2θ log p(θ | D) |θMAP
▶ Now we can sample from the posterior to make predictions:
Z
p(yk | x, D[S] ) ≈ ϕk (fθ (x)) N(θ | θMAP , H−1 ) dθ. (17)
θ

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 59 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Probabilistic Source Hypothesis

▶ Computing H for very deep neural networks (e.g., ResNet-50 (He et al., 2016))
can be infeasible.
▶ Simplification is made by applying Bayesian treatment only to the last-layer (or
hypothesis) of fθ with last-layer Laplace approximation (Kristiadi et al., 2020).

Figure 31: Illustration of source training phase

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 60 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Uncertainty-guided Target Adaptation

▶ As exact solution to the integral in
Eq. (17) is not known, we use Monte
Carlo integration to sample
θj ∼ N(θj | θMAP , H−1 ):

M
1 X  
p(yk | z[T] ) ≈ ϕk hθj (z[T] ) ,
M
j=1
(18)
Figure 32: Feature extractor g is trained under
▶ Uncertainty weights are computed as
a uncertainty-aware composite loss that weighs
samples according to predictive uncertainty wi = exp(−H), where H is entropy.
K
Lug
X
ent = −Ep(x[T] ) w ϕk (f (x[T] )) log ϕk (f (x[T] )). (19)
k=1

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 61 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results under Mild Domain-Shift

Source Target SFDA (IM)

MAP
Uncertainty-
guided

Figure 33: Comparison of conventional IM (MAP) with our uncertainty-guided IM. The solid
vs. hollow circles represent the source and the target data, respectively
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 62 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results under Strong Domain-Shift

Source Target SFDA (IM)

MAP
Uncertainty-
guided

Figure 34: Under strong domain-shift, IM, when used with a MAP estimate, finds a completely
flipped decision boundary. U-SFAN finds the decision boundary by down-weighting the far away
target data
Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 63 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Office-Home (Closed-Set)

Methods A→C A→P A→R C→A C→P C→R P→A P→C P→R R→A R→C R→P Avg.
Source Only 34.9 50.0 58.0 37.4 41.9 46.2 38.5 31.2 60.4 53.9 41.2 59.9 46.1
DANN (Ganin et al., 2016) 45.6 59.3 70.1 47.0 58.5 60.9 46.1 43.7 68.5 63.2 51.8 76.8 57.6
DWT-MEC (Roy et al., 2019) 50.3 72.1 77.0 59.6 69.3 70.2 58.3 48.1 77.3 69.3 53.6 82.0 65.6
CDAN (Long et al., 2018) 50.7 70.6 76.0 57.6 70.0 70.0 57.4 50.9 77.3 70.9 56.7 81.6 65.8
SAFN (Xu et al., 2019) 52.0 71.7 76.3 64.2 69.9 71.9 63.7 51.4 77.1 70.9 57.1 81.5 67.3
LSC (Yang et al., 2021) 57.9 78.6 81.0 66.7 77.2 77.2 65.6 56.0 82.2 72.0 57.8 83.4 71.3
SHOT-IM (Liang et al., 2020) 55.4 76.6 80.4 66.9 74.3 75.4 65.6 54.8 80.7 73.7 58.4 83.4 70.5
U-SFAN 58.5 78.6 81.1 66.6 75.2 77.9 66.3 57.9 80.6 73.6 61.4 84.1 71.8
A2 Net (Xia et al., 2021) 58.4 79.0 82.4 67.5 79.3 78.9 68.0 56.2 82.9 74.1 60.5 85.0 72.8
SHOT (Liang et al., 2020) 57.1 78.1 81.5 68.0 78.2 78.1 67.4 54.9 82.2 73.3 58.8 84.3 71.8
U-SFAN+ 57.8 77.8 81.6 67.9 77.3 79.2 67.2 54.7 81.2 73.3 60.3 83.9 71.9
SHOT++ (Liang et al., 2021) 57.9 79.7 82.5 68.5 79.6 79.3 68.5 57.0 83.0 73.7 60.7 84.9 73.0

Table 13: Comparison with the SFDA state-of-the-art onOffice-Home for the closed-set setting.
A2 Net (Xia et al., 2021) and SHOT++ (Liang et al., 2021) use a multitude of training losses

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 64 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Visda-C and Domain Net (Closed-Set)

Methods Acc.
Source SHOT-IM (Liang et al., 2020) U-SFAN
Source Only 52.4
clipart 25.04 30.88
CDAN+BSP (Chen et al., 2019a) 75.9
infograph 21.58 26.44
SAFN (Xu et al., 2019) 76.1 painting 23.89 29.91
SHOT-IM (Liang et al., 2020) 80.3 quickdraw 10.76 10.44
real 21.74 29.32
U-SFAN 81.2 sketch 28.87 29.99
3C-GAN (Li et al., 2020) 81.6 Avg. 21.98 26.13
A2 Net (Xia et al., 2021) 84.3
Table 15: Comparison with the SFDA
SHOT (Liang et al., 2020) 82.4
state-of-the-art on Domain Net for the
U-SFAN+ 82.7
closed-set setting. The Source column indicates
Table 14: Comparison with the SFDA the domain where the source model has been
state-of-the-art on Visda-C dataset for the trained.
closed-set setting

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 65 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Results on Office-Home (Open-Set)

Methods A→C A→P A→R C→A C→P C→R P→A P→C P→R R→A R→C R→P Avg.
Source Only 53.4 52.7 51.9 69.3 61.8 74.1 61.4 64.0 70.0 78.7 71.0 74.9 65.3
ATI-λ (Busto and Gall, 2017) 55.2 52.6 53.5 69.1 63.5 74.1 61.7 64.5 70.7 79.2 72.9 75.8 66.1
OpenMax (Bendale and Boult, 2016) 56.5 52.9 53.7 69.1 64.8 74.5 64.1 64.0 71.2 80.3 73.0 76.9 66.7
STA (Liu et al., 2019) 58.1 53.1 54.4 71.6 69.3 81.9 63.4 65.2 74.9 85.0 75.8 80.8 69.5
SHOT-IM (Liang et al., 2020) 62.5 77.8 83.9 60.9 73.4 79.4 64.7 58.7 83.1 69.1 62.0 82.1 71.5
SHOT (Liang et al., 2020) 64.5 80.4 84.7 63.1 75.4 81.2 65.3 59.3 83.3 69.6 64.6 82.3 72.8
U-SFAN 62.9 77.9 84.0 67.9 74.6 79.6 68.8 61.3 83.3 76.0 63.9 82.3 73.5

Table 16: Comparison with the SFDA state-of-the-art on Office-Home for the open-set setting.
Open-set (OS) classification accuracy, which also includes the accuracy on the unknown class,
has been reported

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 66 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Overview

1 The Need to Adapt Neural Networks

2 Unsupervised Domain Adaptation

3 Contributions

4 Conclusions

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 67 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Conclusions

▶ Domain Whitening Transform layers to align the marginal feature distributions

between the source and target domains.
▶ TriGAN, a generative framework for multi-source domain adaptation, that
translates source domain images into target-like source images.
▶ Curriculum Graph Co-teaching framework for feature aggregation with a GCN
that aligns similar samples from different domains in order to obtain a unified
feature space in MTDA.
▶ Bayesian framework U-SFAN that constructs a probabilistic source model to
quantify the uncertainty in the source model predictions on the target data and
utilizes it to guide the adaptation.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 68 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Future Works

▶ Domains encountered in the real-world are non-static . Adapting online to such

continually evolving domains (Volpi et al., 2022; Wang et al., 2022) is an open
research question.
▶ Federated domain adaptation (Yao et al., 2022) is an interesting setting where
data on each client machine comprise of an unlabelled target domain.
▶ Humans in the loop for adaptation, (a.k.a active domain adaptation (Prabhu
et al., 2021)) can offer high performing and cost-effective solutions.
▶ Adapting black-box models (Liang et al., 2022) to a target dataset at hand
requires re-thinking of many existing DA strategies.
▶ Given the success of diffusion models (Rombach et al., 2021), we can consider
creating target domain-prompted counterfactuals (Calderon et al., 2022).

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 69 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Roadmap of Ph.D.
2018 2019 2020 2021 2022/23

Under
CVPR ICIAP MVA CVPR ECCV WACV
Review
(DWT) (F2WCT) (TriGAN) (CGCT) (U-SFAN) (CoaST)
(WACV)

Domain Adaptation
Under
Review
Start (ICLR)
ECCV
(FRoST)
Continual Learning
CVPR
(NCL)
November
Novel Class Discovery

Legends

In Thesis ICIAP TMI ICPR

(NAS) (LUS) (CoSeg)

Not in Thesis
Miscellaneous

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 70 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Publications
1. Yangsong Zhang, Subhankar Roy, Hongtao Lu, Elisa Ricci, Stéphane Lathuilière.
“Cooperative Self-Training for Multi-Target Adaptive Semantic Segmentation”. In
WACV, 2023.
2. Subhankar Roy, Mingxuan Liu, Zhun Zhong, Nicu Sebe, Elisa Ricci.
“Class-incremental Novel Class Discovery”. In ECCV, 2022.
3. Subhankar Roy, Martin Trapp, Andrea Pilzer, Juho Kannala, Nicu Sebe, Elisa
Ricci, Arno Solin. “Uncertainty-guided Source-free Domain Adaptation”. In
ECCV, 2022.
4. Subhankar Roy, Evgeny Krivosheev, Zhun Zhong, Nicu Sebe, Elisa Ricci.
“Curriculum Graph Co-Teaching for Multi-Target Domain Adaptation”. In CVPR,
2021.
5. Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, Nicu Sebe.
“Neighborhood Contrastive Learning for Novel Class Discovery”. In CVPR, 2021.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 71 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Publications

6. Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu
Sebe, Elisa Ricci. “TriGAN: Image-to-Image Translation for Multi-Source Domain
Adaptation”. In MVAP, 2021.
7. Aliaksandr Siarohin, Subhankar Roy, Stéphane Lathuilière, Sergey Tulyakov, Elisa
Ricci, Nicu Sebe. “Motion-supervised Co-Part Segmentation”. In ICPR, 2021.
8. Subhankar Roy, Willi Menapace, Sebastiaan Oei, Ben Luijten, Enrico Fini,
Cristiano Saltori, Iris Huijben, Nishith Chennakeshava, Federico Mento,
Alessandro Sentelli, Emanuele Peschiera, Riccardo Trevisan, Giovanni Maschietto,
Elena Torri, Riccardo Inchingolo, Andrea Smargiassi, Gino Soldati, Paolo Rota,
Andrea Passerini, Ruud JG Van Sloun, Elisa Ricci, Libertario Demi. “Deep
learning for classification and localization of COVID-19 markers in point-of-care
lung ultrasound”. In TMI, 2020.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 72 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Publications

9. Subhankar Roy, Aliaksandr Siarohin, Nicu Sebe. “Unsupervised Domain

Adaptation Using Full-Feature Whitening and Colouring”. In ICIAP, 2019.
10. Cristiano Saltori, Subhankar Roy, Nicu Sebe, Giovanni Iacca. “Regularized
Evolutionary Algorithm for Dynamic Neural Topology Search”. In ICIAP, 2019.
11. Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu
Sebe, Elisa Ricci. “Unsupervised Domain Adaptation using Feature-Whitening
and Consensus Loss”. In CVPR, 2019.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 73 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Advisors and Collaborators

Advisors

Prof. Elisa Ricci Prof. Nicu Sebe

Collaborators

Dr. Enver Dr. Aliaksandr Dr. Zhun

Sangineto Siarohin Zhong

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 74 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

Learning to Adapt Neural Networks Across Visual Domains

Ph.D. Defense

Ph.D. Candidate: Subhankar Roy

Advisors: Prof. Elisa Ricci and Prof. Nicu Sebe

Università degli studi di Trento

September 29, 2022

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 75 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References I
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and
Jennifer Wortman Vaughan. A theory of learning from different domains. Machine
learning, 79(1):151–175, 2010.
Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings
of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), pages
1563–1572, 2016.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review
for statisticians. Journal of the American statistical Association, 112(518):859–877,
2017.
Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip
Krishnan. Unsupervised pixel-level domain adaptation with generative adversarial net-
works. In CVPR, 2017.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 76 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References II
Pau Busto and Juergen Gall. Open set domain adaptation. In Proceedings of the IEEE
International Conference on Computer Vision (ICCV), pages 754–763, 2017.
Nitay Calderon, Eyal Ben-David, Amir Feder, and Roi Reichart. Docogen: Domain
counterfactual generation for low resource domain adaptation. In Smaranda Muresan,
Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meet-
ing of the Association for Computational Linguistics (Volume 1: Long Papers), ACL
2022, Dublin, Ireland, May 22-27, 2022, pages 7727–7746. Association for Computa-
tional Linguistics, 2022. URL https://aclanthology.org/2022.acl-long.533.
Fabio Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo. Au-
todial: Automatic domain alignment layers. In Proceedings of the IEEE international
conference on computer vision, pages 5067–5075, 2017.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 77 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References III
Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs.
discriminability: Batch spectral penalization for adversarial domain adaptation. In Pro-
ceedings of the International Conference on Machine Learning (ICML), pages 1081–
1090, 2019a.
Ziliang Chen, Jingyu Zhuang, Xiaodan Liang, and Liang Lin. Blending-target domain
adaptation by adversarial meta-adaptation networks. In Proc. CVPR, 2019b.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul
Choo. StarGAN: Unified generative adversarial networks for multi-domain image-to-
image translation. In CVPR, 2018.
Gabriela Csurka. A comprehensive survey on domain adaptation for visual applications.
Domain Adaptation in Computer Vision Applications, pages 1–35, 2017.
Geoff French, Michal Mackiewicz, and Mark Fisher. Self-ensembling for visual domain
adaptation. ICLR, 2018.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 78 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References IV
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing
model uncertainty in deep learning. In Proceedings of the International Conference
on Machine Learning (ICML), pages 1050–1059, 2016.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle,
François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial train-
ing of neural networks. Journal of Machine Learning Research, 17(59):1–35, 2016.
Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, David Balduzzi, and Wen Li.
Deep reconstruction-classification networks for unsupervised domain adaptation. In
ECCV, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for
image recognition. In CVPR, 2016.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 79 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References V
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield
high-confidence predictions far away from the training data and how to mitigate the
problem. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), pages 41–50, 2019.
Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko,
Alexei A Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain
adaptation. In ICML, 2017.
Haoshuo Huang, Qixing Huang, and Philipp Krahenbuhl. Domain transfer through deep
activation matching. In ECCV, 2018.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network
training by reducing internal covariate shift. In ICML, 2015.
Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion
for versatile domain adaptation. In Proc. ECCV, 2020.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 80 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References VI
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being Bayesian, even just a bit,
fixes overconfidence in relu networks. In Proceedings of the International Conference
on Machine Learning (ICML), pages 5436–5446, 2020.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable
predictive uncertainty estimation using deep ensembles. In Advances in Neural Infor-
mation Processing Systems (NeurIPS), 2017.
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation:
Unsupervised domain adaptation without source data. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), pages 9641–9650,
2020.
Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang. Uni-
versal style transfer via feature transforms. Advances in neural information processing
systems, 30, 2017.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 81 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References VII
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data?
source hypothesis transfer for unsupervised domain adaptation. In Proceedings of the
International Conference on Machine Learning (ICML), pages 6028–6039, 2020.
Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent
unsupervised domain adaptation through hypothesis transfer and labeling transfer.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Jian Liang, Dapeng Hu, Jiashi Feng, and Ran He. Dine: Domain adaptation from single
and multiple black-box predictors. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, pages 8003–8013, 2022.
Hong Liu, Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Qiang Yang. Separate
to adapt: Open set domain adaptation via progressive separation. In Proceedings of
the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages
2927–2936, 2019.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 82 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References VIII
Ming-Yu Liu and Oncel Tuzel. Coupled generative adversarial networks. In NeurIPS,
2016.
Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation
networks. In NeurIPS, 2017.
Mingsheng Long and Jianmin Wang. Learning transferable features with deep adaptation
networks. In ICML, 2015.
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning
with joint adaptation networks. ICML, 2017.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional
adversarial domain adaptation. arXiv:1705.10667v2, 2018.
Zak Murez, Soheil Kolouri, David Kriegman, Ravi Ramamoorthi, and Kyungnam Kim.
Image to image translation for domain adaptation. In CVPR, 2018.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 83 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References IX
Radford M Neal. Bayesian Learning for Neural Networks. Springer Science & Business
Media, 2012.
Le Thanh Nguyen, Madhu Kiran, Jose Dolz, Eric Granger, Atif Bela, and Louis-Antoine
Blais-Morin. Unsupervised multi-target domain adaptation through knowledge distil-
lation. In Proc. WACV, 2021.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Mo-
ment matching for multi-source domain adaptation. ICCV, 2019a.
Xingchao Peng, Zijun Huang, Ximeng Sun, and Kate Saenko. Domain agnostic learning
with disentangled representations. In ICML, 2019b.
Viraj Prabhu, Arjun Chandrasekaran, Kate Saenko, and Judy Hoffman. Active domain
adaptation via clustering uncertainty-weighted embeddings. In Proceedings of the
IEEE/CVF International Conference on Computer Vision, pages 8505–8514, 2021.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 84 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References X
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models, 2021.
Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, and
Elisa Ricci. Unsupervised domain adaptation using feature-whitening and consensus
loss. CVPR, 2019.
Paolo Russo, Fabio Maria Carlucci, Tatiana Tommasi, and Barbara Caputo. From source
to target and back: symmetric bi-directional adaptive gan. In CVPR, 2018.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum
classifier discrepancy for unsupervised domain adaptation. In Proc. CVPR, 2018.
Swami Sankaranarayanan, Yogesh Balaji, Carlos D Castillo, and Rama Chellappa. Gen-
erate to adapt: Aligning domains using generative adversarial networks. In CVPR,
2018.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 85 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References XI
Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening and Coloring batch
transform for GANs. In ICLR, 2019.
Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adap-
tation. In AAAI, 2016.
Luke Tierney and Joseph B Kadane. Accurate approximations for posterior moments and
marginal densities. Journal of the American Statistical Association, 81(393):82–86,
1986.
Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In CVPR, 2011.
Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. Simultaneous deep transfer
across domains and tasks. In ICCV, 2015.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative
domain adaptation. In CVPR, 2017.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 86 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References XII
Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. Instance normalization: The
missing ingredient for fast stylization. arXiv:1607.08022, 2016.
Riccardo Volpi, Pau De Jorge, Diane Larlus, and Gabriela Csurka. On the road to
online adaptation for semantic image segmentation. In Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pages 19184–19195, 2022.
Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain
adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, pages 7201–7211, 2022.
Haifeng Xia, Handong Zhao, and Zhengming Ding. Adaptive adversarial network for
source-free domain adaptation. In Proceedings of the IEEE International Conference
on Computer Vision (ICCV), pages 9010–9019, 2021.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 87 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References XIII
Ruijia Xu, Ziliang Chen, Wangmeng Zuo, Junjie Yan, and Liang Lin. Deep cocktail
network: Multi-source unsupervised domain adaptation with category shift. In CVPR,
2018.
Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. Larger norm more transferable: An
adaptive feature norm approach for unsupervised domain adaptation. In Proceedings
of the IEEE International Conference on Computer Vision (ICCV), pages 1426–1435,
2019.
Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Gen-
eralized source-free domain adaptation. In Proceedings of the IEEE International
Conference on Computer Vision (ICCV), pages 8978–8987, 2021.
Xu Yang, Cheng Deng, Tongliang Liu, and Dacheng Tao. Heterogeneous graph attention
network for unsupervised multiple-target domain adaptation. TPAMI, 2020.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 88 / 89
 The Need to Adapt Neural Networks Unsupervised Domain Adaptation Contributions Conclusions References

References XIV
Chun-Han Yao, Boqing Gong, Hang Qi, Yin Cui, Yukun Zhu, and Ming-Hsuan Yang.
Federated multi-target domain adaptation. In Proceedings of the IEEE/CVF Winter
Conference on Applications of Computer Vision, pages 1424–1433, 2022.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image
translation using cycle-consistent adversarial networks. In CVPR, 2017.

Ph.D. Candidate: Subhankar Roy Advisors: Prof. Elisa Ricci and Prof.
Università
Nicu Sebedegli studi di Trento 89 / 89