WideResNet implementation doesn't use Dropout

[BlackHC]

🐛 Bug

WideResNets as described in the original paper ("Wide Residual Networks" https://arxiv.org/pdf/1605.07146.pdf) use Dropout. The Torchvision implementation does not.

If this for performance reasons, or similar, please add a comment. It would be very helpful to point out that difference clearly. When it comes to reproducing baselines, confusion about what is a real WideResNet vs not could complicate research efforts.

Thanks for your consideration!

To Reproduce

Steps to reproduce the behavior:

Examine model definition in https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py and compare to paper, or other implementations eg https://github.com/meliketoy/wide-resnet.pytorch/blob/master/networks/wide_resnet.py.

Expected behavior

Dropout layers to be added in the residual blocks.

Environment

torchvision 0.6/head

[zhangguanheng66]

@fmassa It seems a reasonable request.

[fmassa]

WideResNet was implemented in torchvision by @szagoruyko (who wrote WideResNet) in #912.

@szagoruyko any comments on that?

[szagoruyko]

Dropout is only used in CIFAR models, there is no dropout in ImageNet models which are exactly the same as ResNet in torchvision except for width, see the original implementation https://github.com/szagoruyko/wide-residual-networks/blob/master/pretrained/wide-resnet.lua

[fmassa]

Closing following @szagoruyko comments

[BlackHC]

Cool, thank you @szagoruyko for sharing this!

So I suppose, it's best to ignore the dropout sections in your paper, as they are not part of the pretrained/sanctioned code for ImageNet and CIFAR10 (as one can tell from reading the Lua and PyTorch code)?

The only use of Dropout I found in https://github.com/szagoruyko/wide-residual-networks/blob/master/models/wide-resnet.lua.

Just in order to clarify that the PyTorch version can be used for baselines etc without having to worry that I'd be misrepresenting anything.

Thanks,
Andreas