Papers Notes_5_ GAN--Generative Adversarial Nets
Adversarial Nets
a discriminative model G + a generative model D
G captures the data distribution
(the generative model generates samples by passing random noise through a multilayer perceptron)
D estimates the probability that a sample came from the training data rather than G
unique solution→
G recovers the training data distribution
D is equal to 1/2 everwhere
to learn the generator’s distribution
p
g
p_g
pg over data
x
x
x
define a prior on input noise variable
p
z
(
z
)
p_z(z)
pz(z)
represent a mapping to data sapce as
G
(
z
;
θ
g
)
G(z;\theta_g)
G(z;θg)
define
D
(
x
;
θ
d
)
D(x;\theta_d)
D(x;θd) outputs a single scalar, the probability that
x
x
x came from the data rather than
p
g
p_g
pg
train
D
D
D to maximize the probability of assigning the correct label to both training examples and examples ffrom
G
G
G
simultaneously train
G
G
G to minimize
l
o
g
(
1
−
D
(
G
(
z
)
)
)
log(1-D(G(z)))
log(1−D(G(z)))
value function
V
(
D
,
G
)
V(D,G)
V(D,G):
min
G
max
D
(
D
,
G
)
=
E
x
∼
p
d
a
t
a
(
x
)
[
l
o
g
D
(
x
)
]
+
E
z
∼
p
z
(
z
)
[
l
o
g
(
1
−
D
(
G
(
z
)
)
)
]
\min \limits_G \max \limits_D(D,G)=\mathbb E_{x\sim p_{data}(x)}[logD(x)]+\mathbb E_{z\sim p_z(z)}[log(1-D(G(z)))]
GminDmax(D,G)=Ex∼pdata(x)[logD(x)]+Ez∼pz(z)[log(1−D(G(z)))]
Training
Early in learning, maximize l o g ( G ( z ) ) log(G(z)) log(G(z)) rather than minimize l o g ( 1 − D ( G ( z ) ) ) log(1-D(G(z))) log(1−D(G(z))) provides much more stronger gradients
Optimizing D to completion in the inner loop of training is computationally prohibtive, and on finite datasets would result in overfitting.
Instead, we alternate between k steps of optimizing D and one step of optimizing G
the generator net used a mixture of rectifier linear activations and sigmoid activations
the discriminator used maxout activations and dropout
use noise as the input to only the bootommost layer of the generator network
References
Generative Adversarial Nets
All code and hyperparameters available at http://www.github.com/goodfeli/adversarial