G. B.

Pant Engineering College, New Delhi

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

SYNOPSIS of Major Project Jan-June 2022

On
Image Super Resolution

B. Tech. (CSE -8thSemester)

SUBMITTED BY: - SUBMITTED TO: -

Shaurya Lalwalia (04020902718) Name of Guide – Dr. Anu Saini
Mudit Pandey (02920902718)
 Statement of the problem

To recover or restore high resolution image from low resolution image. This project aims towards
enhancing low resolution images by applying deep network with adversarial network (Generative
Adversarial Networks) to produce high resolutions images. Our main target is to reconstruct high-
resolution image by developing the low-resolution image such that main details in the reconstructed
images is not lost.

Why a particular topic chosen/Motivation

The motivation behind this project is to develop a high-quality image from a low-resolution image.
Sometimes an image is blurred or distorted because of which main purpose of the image is not
fulfilled. It cannot convey the message it is supposed to.
It has many applications and can be used in numerous fields. For example, an image of a person
from a surveillance camera is not that clear but to know exactly who that person is we need a high-
quality image. Similarly, recovered images from a lost drive/device is recovered in low resolution
which lacks clarity. So, this project will serve the purpose to do so.

Overview of Project

Reconstructing low resolution images to high resolution images by using deep neural network
techniques are quite challenging but can be used in many applications like medical imaging, public
surveillance, or old photo recovery. In comparison with traditional methods, deep learning is more
efficient in generating high resolution images with high accuracy and with greater speed. This
project aims towards enhancing low resolution images by applying Generative Adversarial
Networks to produce high resolutions images. Our main target is to reconstruct high-resolution
image by developing the low-resolution image such that essential details in the reconstructed
images is not lost. GAN has showcased major improvements in the field of generative modeling
problems and image resolution.

Objective and Scope of the project

To enhance a low-resolution image by applying deep network with adversarial network (Generative
Adversarial Networks) to produce high resolutions images.
In the present world, we deal with images overload in our day-to-day life, many of which are

[Computer Science & Engineering] Page 2

distorted or blurred or lacks clarity. Image super resolution comes out as a effective solution for
this.

[Computer Science & Engineering] Page 3

Literature Survey

1. Author [1] has suggested the method called SSSR (self-supervised super-resolution) which
is PET dependent on dual GAN that trains the combinedly to generates SR PET images
from unpaired PET inputs in a self-supervised manner. For supervised training, the images
have been used from the BrainWeb database. The proposed network receives low-
resolution PET. The research shows the result that, SSSR appears to be weaker in
comparison with VDSR it is better than classic deblurring. The last is true in light of the fact
that VDSR is fully supervised and based on paired training data sets.

2. Author [2] has suggested the model, super-resolution generative adversarial network
(SRGAN) with the help of deep residual network (ResNet), for generating an image of
super-resolution using GAN. The proposed network can generate photo-realistic images by
upscaling factor ×4. They developed a function to determine loss which comprises both the
content and an adversarial loss. The target of the proposed SRGAN is to overlook
connection and skip diverge from MSE. The dataset used to be experimented on are Set5,
Set14, and BSD100, and the testing part was done on BSD300.

3. Author [3] has suggested a network called coupled GAN (CoGAN) which focuses on
learning about the joint distribution of multi-domain images without any tuple alike images.
The CoGAN network is based on the possibility that deep neural networks get familiar with
a progressive element portrayal. By implementing the layers that decode significant level
semantics in the GANs to share the weights, it powers the GANs to disentangle the high-
level semantics similarly. The layers that decode low-level information then, at that point
map the common portrayal to images in singular spaces for confusing the separate
discriminative models. CoGAN includes a pair of GAN, each is in charge of incorporating
images in a single domain.

4. In [4], author proposed a temporal adaptive network and explore several methods of image
alignment including a spatial alignment network, for learning the temporal dynamics to
enhance video SR. Our proposed models with learned temporal dynamics are
comprehensively evaluated on various video sequences and achieve state-of-the-art SR
results. Both of the temporal adaptation and the spatial alignment modules are manifested to
increase the robustness to complex motion and thus considerably improve SR performance
over their plain counterparts.

5. This paper discusses the various methods for real-time image super resolution. It focuses
majorly on Convolutional Neural Network (CNN) and Generative Adversarial Network
(GAN) based SR techniques. It then performs numerous comparison test to determine the
best possible technique for real time image SR. Because in real time conversion of LR
image to HR image brings in the challenge of how to adapt deep learning-based SR methods
[Computer Science & Engineering] Page 4
 with acceleration strategies to deal with big data situations.

6. Author [6] proposed a method of super-resolution with the help of a deeply-recursive

convolutional network (DRCN). Expanding recursion depth can further develop execution
without presenting new boundaries for extra convolutions. Two methods have been
suggested to facilitate the struggle of training. Firstly, supervision of all the recursion is
required. The reconstruction method is something very similar for all recursions.

7. Author [7] has suggested the model, conditional generative adversarial network (CGAN)
states that methods have no control over the information being created in an unconditioned
generative model. If both the generative model and the discriminative model are adjusted
based on some other information named y which can be any additional data (such as class
labels, etc.), then GAN can be further extended to a conditional model.

Above are some of the examples of the research papers we’ve studied to know and understand the
different approaches to implement image super resolution. The networks discussed in the research
papers optimize the pixel difference between predicted and output HR images. Generative models
(or GANs) try to optimize the perceptual quality to produce images which are pleasant to the human
eye.
We’re specifically using SRGAN which uses a GAN-based architecture to generate visually
pleasing images. It uses the SRResNet network architecture as a backend, and employs a multi-task
loss to refine the results.

Methodology

GAN is a generative deep learning algorithm that creates new data instances that resembles the
training data. GAN has two components (generator and discriminator) both compete with each other
resulting in mutual comparison. Generator’s function is to learn to generate fake data whereas
discriminator learns from the false information generated. The generator tries to mislead
discriminator by creating fake inputs and tries to fool discriminator thinking these are real inputs.
Discriminator tells us that if an input is real or fake. The networks discussed so far optimize the
pixel difference between predicted and output HR images. Generative models (or GANs) try to
optimize the perceptual quality to produce images which are pleasant to the human eye.

We’re specifically using SRGAN which uses a GAN-based architecture to generate visually
pleasing images. It uses the SRResNet network architecture as a backend, and employs a multi-task
loss to refine the results.

[Computer Science & Engineering] Page 5

Hardware and Software Requirement
 Software
Pytorch/AWS/jupyter

 Language
Python

References

1) T.-A. Song, S. Chowdhury, K. Kim, K. Gong, G. El Fakhri, Q. Li, and J. Dutta, “Super-
resolution PET using a very deep convolutional neural network,” in Proc IEEE Nucl Sci
Symp Med Imag Conf. IEEE, 2018.

2) C. Ledig, L. Theis, F. Huszar, J. Caballero, A. P. Aitken, A. Tejani, J. Totz, Z. Wang, and W.

Shi, “Photo-realistic single image super-resolution using a generative adversarial network,”
CoRR, vol. abs/1609.04802, 2016.

3) M.-Y. Liu and O. Tuzel. “Coupled generative adversarial networks”. In Advances in Neural
Information Processing Systems (NIPS), 2016.

4) Ding Liu, Student Member, IEEE, Zhaowen Wang, Member, IEEE, Yuchen Fan, Xianming
Liu, Zhangyang Wang, Member, IEEE, Shiyu Chang, Xinchao Wang, and Thomas S. Huang,
Life Fellow, IEEE, Learning Temporal Dynamics for Video Super-Resolution: A Deep
Learning Approach, 2018

[Computer Science & Engineering] Page 6

5) Xiaofang Li2 · Yirui Wu1 · Wen Zhang1 · Ruichao Wang3 · Feng Hou4, Deep learning
methods in real-time image super-resolution: a survey, 2019
6) J. Kim, J. K. Lee, and K. M. Lee. “Deeply-recursive convolutional network for image super-
resolution” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
7) M. Mirza and S. Osindero. “Conditional generative adversarial nets”. arXiv preprint
arXiv:1411.1784, 2014.

[Computer Science & Engineering] Page 7

[Computer Science & Engineering] Page 8