Assignment 5: CS 663

Due: 5th November before 11:55 pm

Remember the honor code while submitting this (and every other) assignment. You may discuss
broad ideas with other students or ask me for any difficulties, but the code you implement and the
answers you write must be your own. We will adopt a zero-tolerance policy against any violation.

Submission instructions: Follow the instructions for the submission format and the naming convention of your
files from the submission guidelines file in the homework folder. Please see assignment5 DFT.rar. Upload the file on
moodle before 11:55 pm on 5th November and this is also the cutoff deadline. This is different from other
assignments where you had time till the next day 10 am. But 5th November is the last day of classes.
Only one student per group needs to upload the assignment. No late assignments will be accepted after this time.
Please preserve a copy of all your work until the end of the semester.

1. In this part, we will apply the PCA technique for the task of image denoising. Consider the images barbara256.png
and stream.png present in the corresponding data/ subfolder - this image has gray-levels in the range from
0 to 255. For the latter image, you should extract the top-left portion of size 256 by 256. Add zero mean
Gaussian noise of σ = 20 to one of these images using the MATLAB code im1 = im + randn(size(im))*20.
Note that this noise is image-independent. (If during the course of your implementation, your program takes
too long, you can instead work with the file barbara256-part.png which has size 128 by 128 instead of 256 by
256. You can likewise extract the top-left 128 by 128 part of the stream.png image. You will not be penalized
for working on these image parts.)

(a) In the first part, you will divide the entire noisy image ‘im1’ into overlapping patches of size 7 by 7, and
create a matrix P of size 49 × N where N is the total number of image patches. Each column of P
is a single patch reshaped to form a vector. Compute eigenvectors of the matrix PPT , and the eigen-
coefficients of each noisy patch. P Let us denote the j th eigen-coefficient of the ith (noisy) patch (i.e. Pi )
by αij . Define ᾱj = max(0, N [ N
2 1 2 2
i=1 αij ] − σ ), which is basically an estimate of the average squared
eigen-coefficients of the ‘original (clean) patches’. Now, your task is to manipulate the noisy coefficients
{αij } using the following rule, which is along the lines of the Wiener filter update that we studied in class:
denoised = αij denoised stands for the j th eigencoefficient of the ith denoised patch. Note that
αij σ2
. Here, αij
1 + ᾱ2
j
σ2
ᾱ2j
is an estimate of the ISNR, which we absolutely need for any practical implementation of a Wiener
filter update. After updating the coefficients by the Wiener filter rule, you should reconstruct the denoised
patches and re-assemble them to produce the final denoised image which we will call ‘im2’. Since you chose
overlapping patches, there will be multiple values that appear at any pixel. You take care of this situation
using simple averaging. Write a function myPCADenoising1.m to implement this. Display the final image
kim2denoised − im2orig k2
‘im2’ in your report and state its RMSE computed as .
kim2orig k2
(b) In the second part, you will modify this technique. Given any patch Pi in the noisy image, you should
collect K = 200 most similar patches (in a mean-squared error sense) from within a 31 × 31 neighborhood
centered at the top left corner of Pi . We will call this set of similar patches as Qi (this set will of course
include Pi ). Build an eigen-space given Qi and denoise the eigen-coefficients corresponding to only Pi
using the Wiener update mentioned earlier. The only change will be that ᾱj2 will now be defined using only
the patches from Qi (as opposed to patches from all over the image). Reconstruct the denoised version of
Pi . Repeat this for every patch from the noisy image (i.e. create a fresh eigen-space each time). At any
pixel, there will be multiple values due to overlapping patches - simply average them. Write a function
 myPCADenoising2.m to implement this. Reconstruct the final denoised image, display it in your report
and state the RMSE value. Do so for both barbara as well as stream.
(c) Now run your bilateral filter code from Homework 2 on the noisy version of the barbara image. Compare
the denoised result with the result of the previous two steps for both images. What differences do you
observe? What are the differences between this PCA based approach and the bilateral filter?
(d) Consider that a student clamps the values in the noisy image‘im1’ to the [0,255] range, and then denoises
it using the aforementioned PCA-based filtering technique which assumes Gaussian noise. Is this approach
correct? Why (not)? [10 + 20 + 5 + 5 = 40 points]

2. Read Section 1 of the paper ‘An FFT-Based Technique for Translation, Rotation, and Scale-Invariant Image
Registration’ published in the IEEE Transactions on Image Processing in August 1996. A copy of this paper is
available in the homework folder.

(a) Describe the procedure in the paper to determine translation between two given images. What is the time
complexity of this procedure to predict translation if the images were of size N × N ? How does it compare
with the time complexity of pixel-wise image comparison procedure for predicting the translation?
(b) Also, briefly explain the approach for correcting for rotation between two images, as proposed in this paper
in Section II. Write down an equation or two to illustrate your point.

[15+15=30 points]

3. Consider a matrix A of size m × n, m ≤ n. Define P = AT A and Q = AAT . (Note: all matrices, vectors and
scalars involved in this question are real-valued).

(a) Prove that for any vector y with appropriate number of elements, we have y t P y ≥ 0. Similarly show
that z t Qz ≥ 0 for a vector z with appropriate number of elements. Why are the eigenvalues of P and Q
non-negative?
(b) If u is an eigenvector of P with eigenvalue λ, show that Au is an eigenvector of Q with eigenvalue λ. If v
is an eigenvector of Q with eigenvalue µ, show that AT v is an eigenvector of P with eigenvalue µ. What
will be the number of elements in u and v?
AT v i
(c) If v i is an eigenvector of Q and we define ui , . Then prove that there will exist some real,
kAT v i k2
non-negative γi such that Aui = γi v i .
(d) It can be shown that uTi uj = 0 for i 6= j and likewise v Ti v j = 0 for i 6= j for correspondingly distinct
eigenvalues.1 . Now, define U = [v 1 |v 2 |v 3 |...|v m ] and V = [u1 |u2 |u3 |...|um ]. Now show that A = U ΓV T
where Γ is a diagonal matrix containing the non-negative values γ1 , γ2 , ..., γm . With this, you have just
established the existence of the singular value decomposition of any matrix A. This is a key result in linear
algebra and it is widely used in image processing, computer vision, computer graphics, statistics, machine
learning, numerical analysis, natural language processing and data mining. [7.5 + 7.5 + 7.5 + 7.5 = 30
points]

1
This follows because P and Q are symmetric matrices. Consider P u1 = λ1 u1 and P u2 = λ2 u2 . Then uT2 P u1 = λ1 uT2 u1 . But
uT2 P u1 also equal to (P T u2 )T u1 = (P u2 )T u1 = (λ2 u2 )T u1 = λ2 uT2 u1 . Hence λ2 uT2 u1 = λ1 uT2 u1 . Since λ2 6= λ1 , we must have
uT2 u1 = 0.