Universidad Central “Marta Abreu” de Las Villas

Departamento de Automática y Sistemas Computacionales

Computer Vision
Theme 3
Image Segmentation
2021
Rubén Orozco-Morales [email protected]

Due date: July 5, 2021 at 12:00 PM.

Warm: Read “Assignment Guidelines”.
Advise: After read these assignments, immediately e-mail the professor to reserve the
assignment of your choose. Pleas, note that there will only be one student per
assignment.

Assignments
Assignment # 1:
1) Among the available schemes, Active Contour Models (ACMs), based on evolution curves, are
widely applied to image segmentation, scene understanding and target tracking, etc. The main
idea is deformation of the initial curves, which are induced to move towards the boundaries of
the objects in the image. Your written argumentation consists of doing a literature review on
recent (post 2017) developments and applications related to ACM-based image segmentation
methods.
2) The Matlab-2017 function called activecontour.m implements the classical ACM of T.F. Chan
and L.A. Vese published in 2001 in the IEEE Transactions on Image Processing under the
paper name “Active contours without edges”. Your programming walkthroughs consist of
investigating and demonstrating the full capabilities of Matlab-2017 for ACM-based image
segmentation. For this, use the images 24077.jpg, 37073.jpg, 69020.jpg, 106024.jpg,
227092.jpg and Focused41.jpg that are in the “ImageSet_Theme3” folder. For the first five
images compute the objective segmentation quality (full reference) using the
“Segmentat_QualityAssess_Toolbox”. To do this, you can compare the edges computed by
activecontour.m on the original image and on your ground truth image (see papers in toolbox
designated for quiz 3) in this assignment). However, you can also use the imfill.m Matlab-2017
function to fill the segmented image using activecontour.m function and afterward compute the
segmentation quality. These images and quality toolbox can be downloaded from the Computer
Vision course page on Moodle. Then, write a report with your work, programs and results.

 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

3) A nonparametric region-based AC model for segmenting cluttered scenes was published in

2009 by K. Ni, X. Bresson, T. Chan, and S. Esedoglu in the International Journal of Computer
Vission with the title “Local Histogram Based Segmentation Using the Wasserstein Distance”.
This AC method has been programed in Matlab-2017 and can be downloaded from the
“Segmentat_ActiveContour_Ni09_Toolbox” folder on the Computer Vision course page in
Moodle. Your programming challenge is an evaluation of this toolbox using the same images
and quality assessment performed in quiz 2) of this assignment. Write a report detailing your
work and showing the results.

Assignment # 2:
1) Differential Evolution (DE) is a population-based evolutionary optimization technique that
mimics the evolutionary process in nature. Its procedure is simple, but very effective in solving
various real-life problems, and follows four steeps: initialization, mutation, crossover, and
selection. However, the DE has been modified and extended several times with various versions
being proposed. Your written argumentation consists of doing a literature review on recent (post
2017) developments and applications related to DE-based image segmentation methods.
2) Differential Evolution is an evolutionary data clustering. Clustering itself can be posed as an
optimization problem, so it can be solved using optimization algorithms, specially using
evolutionary and metaheuristic algorithms. Also, an extended version of clustering task, known
as Automatic Clustering, can be easily solved using evolutionary methods. A simple Matlab-
2017 implementation of the classical DE algorithm for color clustering can be downloaded from
the “Clustering_DiffEvol_Color_Toolbox” folder on the Computer Vision course page in
Moodle. Your programming walkthroughs consist first of an analysis of the demo, functions
and documents in this toolbox. Next, run the demonstrative and investigate how the parameters
can influence the performance of the algorithm when the number of classes is two and when is
five. To do this, use the images 65019.jpg, 124084.jpg, 181079.jpg, 288001.jpg, and 388016.jpg
that are in the “ImageSet_Theme3” folder. These images can be downloaded from the Computer
Vision course page on Moodle. Then, write a report with your work, programs and results.
3) Multilevel thresholding plays an important role in partition the image into its meaningful
regions and extract its main features. Using classical implementations for this is time consuming
as they exhaustively search the best values to optimize the objective function. Therefore, the
convergence rate of multilevel thresholding methods have been speeded up by using different
techniques and schemes such as iterative and recursive approaches, including meta-heuristic
algorithms, due to their ability for global optimization. Differential Evolution has been used to
solve this optimization problems, resulting in faster and more accurate convergence to
optimums, as in the work of S. Sarkar et al., presented in 2014 in the International Conference
on Swarm, Evolutionary, and Memetic Computing entitled “A fuzzy entropy based multi-level
image thresholding using differential evolution”. Sarkar’s method, programmed in Matlab-2017
2 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

can be downloaded from the “Clustering_DiffEvol_FuzzEntro_Toolbox” folder on the

Computer Vision course page in Moodle. First analyze the demonstrative, functions and
documents of this toolbox. Next, run the demonstrative and investigate how the parameters can
influence the performance of the algorithm when the number of classes is two and when the the
number of classes is five. To do this, use the images 65019.jpg, 124084.jpg, 181079.jpg,
288001.jpg, and 388016.jpg. Your programming challenge is twofold. The first is to evaluate
objectively (full reference) the quality of the binarization using the
“Segmentat_QualityAssess_Toolbox”, also on the Computer Vision course page in Moodle.
The other challenge is to convert the original RGB image to a luminance-chrominance color
space, such as YCbCr, YUV, HSB, HSL, or Lab, using the colorspace.m function, and then
perform the binarization in the luminance channel. The point is not to use the rgb2gray.m
function of Matlab-2017. Then, objectively evaluate the quality of the binarization for this
alternative approach. Write a report detailing your work and showing the results.

Assignment # 3:
1) Edge detection (ED) is a fundamental step in many computer vision problems involving, among
others, denoising, restoration, and high-level segmentation methods. In these methods proper
edge identification qualifies the performance of subsequent stages. There are diverse ED
methods, basically because edges detection from noisy or corrupted images is a difficult task.
So, your written argumentation consists of doing a literature review on recent (post 2017)
developments and applications related to ED-based image segmentation methods.
2) The of Matlab-2017 function named edge.m implements some classical edge detection methods
in intensity (grayscale) images, namely Sobel, Prewitt, Roberts, Laplacian of Gaussian (LoG),
and Canny methods. Your programming walkthroughs consist of investigating and
demonstrating the full capabilities of Matlab-2017 for ED-based image segmentation. To do
this, use the images 124084.jpg, 181079.jpg, 153093.jpg, 227092.jpg and 376043.jpg that can
be found in the “ImageSet_Theme3” folder that you can download from the Computer Vision
course page on Moodle. Then, write a report with your work, programs and results.
3) Classical edge detection focuses on finding signal discontinuities in the image that mark the
transition from one region to another, focusing on low-level cues, such as pixel intensity or
color. Advanced methods have exploited higher-level representations of images, such as optical
flow, surface and depth cues, as well as object category specific information. In 2014 M.
Leordeanu, R. Sukthankar, and C. Sminchisescu published in IEEE Transactions on Pattern
Analysis and Machine Intelligence the paper entitled “Generalized Boundaries from Multiple
Image Interpretations”. A version of the Generalized Boundaries (GB) algorithm programmed
in Matlab-2017 has been shared for you in the “Edges_GB_Leordeanu14_Toolbox” on the
Computer Vision course page on Moodle. However, the original code can be downloaded from
http://www.imar.ro/clvp/code/Gb. First review the demonstrative, functions and documents in
3 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

this toolbox. Next, try to detect edges on some of the images used in quiz 2). Your programming
challenge include: a) create another demonstrative using the Gb_CS.m function. b) Attempt to
implement a binary image with the close boundary of the main object in each image. c) For
these images compute the objective segmentation quality (full reference) using the
“Segmentat_QualityAssess_Toolbox” which can be downloaded from the Computer Vision
course page on Moodle. Write a report detailing your work and showing the results.

Assignment # 4:
1) The Expectation Maximization (EM) algorithm is an iterative model-based clustering method
for finding maximum likelihood or maximum a posteriori (MAP) estimates of parameters in
statistical models, where the model depends on unobserved latent variables. The EM iteration
alternates between performing an expectation step, which creates a function for the expectation
of the log-likelihood evaluated using the current estimate for the parameters, and a
maximization step, which computes parameters maximizing the expected log-likelihood found
on the expectation step. These parameter-estimates are then used to determine the distribution
of the latent variables in the next expectation step. Compared with other clustering algorithms,
the EM algorithm demands more computational efforts although it produces exceptionally good
results. Consequently, many researches have experimented with some variants such as
Generalized EM (GEM), Expectation Conditional Maximization (ECM), Sparse EM (SpEM),
Lazy EM (LEM), Expectation-Conditional Maximization Either (ECME) algorithm and the
Space Alternating Generalized Expectation (SAGE) maximization algorithms. Then, your
written argumentation consists of doing a literature review on recent (post 2017) developments
and applications related to EM-based image segmentation methods.
2) Expectation Maximization is an effective and popular technique for estimating mixture model
parameters (as in clustering problems). Compared to other clustering algorithms, the EM
algorithm demands more computational efforts but produces exceptionally good results. An
implementation of the EM algorithm in color clustering for digital images can be seen
programmed in Matlab-2017 in the “Clustering_ExpMax_Color_Toolbox” on the Computer
Vision course page in Moodle. Review the demonstratives, functions and documentation in this
toolbox. Your programming walkthroughs consist of investigating the parameter settings and
the difference between the two demos (with and without color shifting), including perceptual
performance, runtime and so on, in both, binarized and multiclass clustering. To do this, use the
images 69020.jpg, 86016.jpg, 124084.jpg, 189080.jpg and 388016.jpg that are in the
“ImageSet_Theme3” folder. Then, write a report with your work, programs and results.
3) Two of the most famous clustering algorithms are expectation maximization (EM) and k-means
(KM). Both algorithms, KM and EM, look in different areas, KM can be considered as an
approximate way to obtain maximum likelihood estimates for the means, which is the goal of
density estimation in EM. Then, in order of speed up the clustering process these two algorithms
4 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

can be combined. A Matlab-2017 implementation of the KM-EM color clustering approach has
been included in the “Clustering_ExpMaxKm_Color_Toolbox” folder on the Computer Vision
course page in Moodle. Review the demonstratives, functions and documents in this toolbox.
Your programming challenge is to attempt to obtain your best binarization with this toolbox
using the same images as in the quiz 2) and assess the quality of binarization using the
“Segmentat_QualityAssess_Toolbox”, also on the Computer Vision course page on Moodle.
Write a report detailing your work and showing the results.

Assignment # 5:
1) The Fuzzy c-means (FCM) algorithm is a clustering technique widely used in image
segmentation, in which cluster formation is controlled by using soft/fuzzy similarity measure.
In FCM the objects that are present at the boundaries between different classes or groups are
not required to be fully associated with one of the classes or a cluster, but rather the membership
degrees are to be assigned between the 0 and 1 which shows their partial membership. However,
its main disadvantage is that it does not take into account any spatial information for noisy in
the images for the fuzzy c-means. To overthrow these imperfections of FCM, many variants
have been introduced such as Improved FCM (IFCM), Possibilistic fuzzy c-means (PCM),
Fuzzy Possibilistic c-means (FPCM), and fuzzy c-means based on histogram. Then, your
written argumentation consists of doing a literature review on recent (post 2017) developments
and applications related to FCM-based image segmentation methods.
2) In the definition of objects based on fuzzy connectivity, a local fuzzy relation called affinity is
defined in the image domain that assigns to each pair of nearby image elements a strength of
local hanging togetherness which has a value in [0, 1]. Also, the affinity between two elements
depends on their spatial nearness as well as how similar their image intensities and intensity-
derived features are. A global fuzzy relation called fuzzy connectedness is defined on the image
domain, which assigns to every pair (c; d) of image elements a strength of global hanging
togetherness that has a value in [0, 1]. These ideas were implemented by P.K. Saha and J.K.
Udupa in 2001 in their paper entitled “Fuzzy Connected Object Delineation: Axiomatic Path
Strength Definition and the Case of Multiple Seeds”, published in Computer Vision and Image
Understanding. You can find a Matlab-2017 version of the Fuzzy Connected Object
Delineation (FCOD) algorithm in the “Segmentat_FCM_FCOD_Saha01_Toolbox” folder on
the Computer Vision course page in Moodle. However, the original toolbox can be downloaded
from https://github.com/joakimlindblad/FuzzConn. Review the demonstratives, functions and
documentation in this toolbox. Your programming walkthroughs is to investigate the parameter
settings and then try to segment the images 24077.jpg, 37073.jpg, 69020.jpg, 106024.jpg and
227092.jpg that are in the “ImageSet_Theme3” folder. You can use more than one seed or use
the ginput.m function to get the seeds. Then, write a report with your work, programs and
results.

5 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

3) The Matlab-2017 function named fcm.m implements the fuzzy c-means clustering method
issued in 1981 by J.C. Bezdec in his work “Pattern Recognition with Fuzzy Objective Function
Algorithms”, appeared in Plenum Press. Your programming challenge is to implement an FCM-
based image color clustering method using the full capabilities of Matlab-2017 for FCM-
clustering. To do this, use the same images as in quiz 2) of this assignment. For these five
images compute the objective segmentation quality (full reference) using the
“Segmentat_QualityAssess_Toolbox”. These images and quality toolbox can be downloaded
from the Computer Vision course page on Moodle. Write a report detailing your work and
showing the results.

Assignment # 6:
1) Gaussian Mixture Models (GMM) are one of the semiparametric techniques for estimating
probability density functions (pdf). In segmentation approaches, GMM assume that the
histogram of pixel intensities in an image is multimodal. That is, the histogram has a finite
number pdf, and each of these distributions can be modeled by a Gaussian. Accordingly, the
whole image can be modeled by a mixture of component Gaussian distributions in some
unknown proportions. The problem is how to set the number of pdf, how to parametrize these
distributions (mean and standard deviation values), as well as how to address the problem of
the joint conditional density of the observations. Additionally, one of the weakness is that noise
not always can be modeled as a Gaussian. Therefore, to obtain more robust performance and
increase segmentation accurately, GMM is combined with, among others, maximum likelihood
estimation or expectation maximization. Then, your written argumentation consists of doing a
literature review on recent (post 2017) developments and applications related to GMM-based
image segmentation methods.
2) Agroundbreaking work by H.A. Phoulady et al., published in 2017 in Computerized Medical
Imaging and Graphics with the title “A framework for nucleus and overlapping cytoplasm
segmentation in cervical cytology extended depth of field and volume images” makes use of
thresholding based on the learned GMM with two components (binarization) on pixel intensities
using the Expectation Maximization (EM) algorithm, followed by morphological operations, to
segment nucleus and overlapping cytoplasm in cervical cytology images. A Matlab-2017
version of Phoulady’s method has been included in the
“Segmentat_GMM_EM_Phoulady17_Toolbox” on the Computer Vision course page in
Moodle. However, the original toolbox can be downloaded from https://github.com/parham-
ap/overlapping_cervical_cell_segmentation. Review the demonstrative, functions and
documentation in this toolbox. Your programming walkthroughs is to do your best trying to
segment the images Erythro0018.tiff, Erythro5817.tiff, papsmear1054.bmp, papsmear1143.png
and papsmear2014.bmp that are in the “ImageSet_Theme3” folder. Note that you can choose

6 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

the luminance of YUV, Lab or HSV color spaces. Then, write a report with your work, programs
and results.
3) The Matlab-2017 function named fitgmdist.m fits the Gaussian mixture distribution to the data
using the method issued in 2000 by G. McLachlan and D. Peel in their work “Finite Mixture
Models”, published by John Wiley & Sons. Your programming challenge is to implement a
GMM-based image color clustering method using the full capabilities of Matlab-2017 for
GMM-distributions. You will need to replace each pixel value with the value of its cluster center
using the posterior.m function. Use the same images as in quiz 2) of this assignment. For these
five images compute the objective segmentation quality (full reference) using the
“Segmentat_QualityAssess_Toolbox”. These images and quality toolbox can be downloaded
from the Computer Vision course page on Moodle. Write a report detailing your work and
showing the results.

Assignment # 7:
1) Genetic Algorithms (GA) has been used for different optimization problems and is part of the
evolutionary algorithms. Evaluation, selection, crossover and mutation are the main operators
of a GA. The optimum solution is obtained by using stochastic operators instead of deterministic
rules, acting directly on binary strings of characters. GAs must meet two main requirements.
First, the problem must be susceptible to genetic representation. This requires that values and
variables be capable of being stored and mutated in a meaningful way. Second, there must be a
fitness function to qualify each solution so that the optimal results can be chosen. Some
downsides are their computationally intensive demands and that they do not work well for
complex systems. In addition, their best solutions have limited creativity, since they are only
the best solution given the fitness function. Therefore, GAs are used to determine the optimal
segmentation based on a genetic representation of possible solutions and a fitness function.
Then, your written argumentation consists of doing a literature review on recent (post 2017)
developments and applications related to GA-based image segmentation methods.
2) In this quiz you will use the crossover and mutation operations to implement in Matlab-2017 a
simple GA algorithm for color clustering. Download the “Clustering_GA_Color_Toolbox”
folder from the Computer Vision course page on Moodle and review the demonstrative and
functions. Your programming walkthroughs consist of explaining the steps of the
GA_ColorClust.m function. After that, try to achieve your best result by running the
demonstrative to cluster the colors of the images Focused05.png, Compos3.png, 37073.jpg,
124084.jpg and 376043.jpg that are in the “ImageSet_Theme3” folder. Then, write a report with
your work, programs and results.
3) The Matlab-2017 function named ga.m find the minimum of a function using GA by the method
issued in 1997 by  R. Conn, N.I.M. Gould, and L. Toint in their work entitled “A Globally

7 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

Convergent Augmented Lagrangian Barrier Algorithm for Optimization with General

Inequality Constraints and Simple Bounds”, published in Mathematics of Computation. Also,
the gaoptimset.m function can create an option structure of the GA and pdist2.m allows to obtain
the pairwise distance between two sets of observations. Firstly, review the help of these three
functions. As you can see, it is somewhat challenging to define a cost function and get a
successful GA clustering scheme. Then, you will get some help. Download the
“Clustering_GA_Matlab_Toolbox” folder from the Computer Vision course page on Moodle
and review the demonstrative, function and documentation in this folder. Now, run the
demonstrative to familiarize yourself with this procedure and try to do your best clustering the
colors of the same images as in the quiz 2) of this assignment. Your programming challenge is
to surprise me by creating at least one new clustering cost function with passable succeed. Write
a report detailing your work and showing the results, including the comparison between
approaches of quizzes 2) and 3).

Assignment # 8:
1) Graph Cut (GC) based algorithms have been shown to be an effective approach for image
segmentation. Among the various graph based approaches, spectral clustering becomes an
important trend. Recent methods include Normalized Cuts (NCut) to segment image into
meaningful partitions. Also, diffusion-based approaches like GrabCut is another region based
segmentation. These have computed more effective affinity matrices using elaborately designed
low-level features and metrics. Furthermore, to address limitations in capturing distant pixel
affinities, several methods based on multiscale affinity strategies have been proposed. However,
dense affinity suffers from an optimization bottleneck, although approximation algorithms are
explored. Even so, several variants of GC for image segmentation have been launched in recent
times. Then, your written argumentation consists of doing a literature review on recent (post
2017) developments and applications related to GC-based image segmentation methods.
2) An efficient segmentation algorithm, based on a predicate for measuring the evidence of a
boundary between two regions, using a graph-based image representation was introduced in
2004 by P.F. Felzenszwalb and D.P. Huttenlocher in the International Journal of Computer
Vision under the title “Efficient Graph-Based Image Segmentation”. A Matlab-2017 version of
this algorithm can be downloaded from the
“Segmentat_EffGraphCut_Felzenszwalb04_Toolbox” folder on the Computer Vision course
page in Moodle. Review the demonstrative, functions and documentation in this folder. Your
programming walkthroughs is to do your best segmenting the images twohorses.png,
5161812.png, 111876273311.png, birdhouse.png and vehicles_land.png that are in the
“ImageSet_Theme3” folder. Then, write a report with your work, programs and results.
3) Matlab-2017 provides the Graph Cut option in the Image Segmenter app. to segment an image
into foreground and background elements, as well as the lazysnapping.m function. Your
8 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

programming challenge is to investigate and demonstrate the full capabilities of Matlab-2017

for GC-based image segmentation. To do so, use the same images as in quiz 2) of this
assignment. Also, compute the objective segmentation quality (full reference) using the
“Segmentat_QualityAssess_Toolbox” that can be downloaded from the Computer Vision
course page on Moodle. Write a report detailing your work and showing the results.

Assignment # 9:
1) The k-means (KM) algorithm is a simple and fast approach for clustering-based image
segmentation. It groups the sample elements into clusters according to the distance of some
randomly distributed k-values to all other elements in the sample to segment the images. A
distinctive of k-means clusters is that items in the same cluster have the shortest lengths from
the k-values in its own cluster other than the k-values that other clusters have. Also, every cluster
has exactly one k-values in it. A shortcoming of the KM algorithm is that it cannot correctly
and automatically generate the value of k (the number clusters into which the image will be
segmented). A random choice of k will result in over- or under-segmentation. Another faults is
that it is sensitive to outliers and prone to local minima. Even so, several variants of KM have
been introduced and abundant application with high performance have been reported. Then,
your written argumentation consists of doing a literature review on recent (post 2017)
developments and applications related to KM-based image segmentation methods.
2) One of the variants of k-means is known as litemameans. It was proposed in 2011 by Cai et al.
in the work “Litekmeans: the fastest Matlab implementation of kmeans”, available in
http://www.zjucadcg.cn/dengcai/Data/Clustering.html. According to Zhao et al. (2015), in their
work “A grid-growing clustering algorithm for geo-spatial data”, published in Pattern
Recognition Letters, litekmeans is the fastest Matlab implementation of the KM algorithm
because it is accelerated by matrix operations. You can find a Matlab-2017 version of
litekmeans in the “Clustering_Litekmeans_Cai11_Toolbox” folder that can be downloaded
from the Computer Vision course page on Moodle. Review the demonstrative, functions and
documents in this folder. Your programming walkthroughs is to do your best clustering the
images 24077.jpg, 37073.jpg, 69020.jpg, 106024.jpg, 227092.jpg and Focused41.jpg that are
in the “ImageSet_Theme3” folder. Note that you can use different color spaces. Try also a
binarization and several multiclass clustering. Then, write a report with your work, programs
and results.
3) The Matlab-2017 function named kmeans.m implements the k-means++ algorithm of D. Arthur
and S. Vassilvitskii published in 2007. Your programming challenge is to investigate and
demonstrate the full capabilities of Matlab-2017 for KM-based image color clustering. To do
this, use the same images as in quiz 2) of this assignment. For them, compute the objective
segmentation quality (full reference) using the “Segmentat_QualityAssess_Toolbox” which can

9 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

be downloaded from the Computer Vision course page on Moodle. Write a report detailing your
work and showing the results.

Assignment # 10:
1) Level Set (LS) methods have been widely used in image processing and computer vision
applications, allowing shape reconstruction, volume estimation, object detection and object
classification. The basic idea of the LS method is to represent a contour as the zero level set of
a higher-dimensional function, called a level set function (LSF), and to formulate the motion of
the contour as the evolution of the LSF. In conventional LS formulations, the LSF often
develops irregularities during its evolution, which can lead to numerical errors and eventually
destroy the stability of the evolution. Therefore, a numerical remedy, called reinitialization, is
usually applied to periodically replace the degraded LSF with a signed distance function. Even
so, the practice of reinitialization not only poses serious problems as to when and how it should
be performed, but also affects numerical accuracy in an undesirable way. Until now, diverse
variants of LS methods have been proposed for image segmentation, and they can currently be
categorized into two major classes: region-based models and edge-based models. Then, your
written argumentation consists on conduct a literature review on recent (post 2017)
developments and applications related to LS-based image segmentation methods.
2) A revolutionary type of LS evolution called Distance Regularized Level Set Evolution
(DRLSE) was proposed in 2010 by C. Li, C. Xu, C. Gui, and M.D. Fox in a paper entitled
“Distance Regularized Level Set Evolution and its Application to Image Segmentation”
published in IEEE Transactions on Image Processing. According to the authors, the distance
regularization effect eliminates the need for reinitialization and thus avoids the numerical errors
induced by reinitialization. Also, in contrast to the complicated implementations of
conventional LS formulations, a simpler and more efficient finite difference scheme can be used
to implement the DRLSE formulation. Additionally, DRLSE also allows the use of a more
general and efficient initialization of the level set function. To verify these points, download a
Matlab-2017 implementation of the DRLSE algorithm from the
“Segmentat_LevelSet_DRLSE_Li11_Toolbox” folder on the Computer Vision course page in
Moodle. Review the demonstratives, functions and documentation in this folder. Your
programming walkthroughs begin by running the three demos and analyzing in each demo how
sensible DRLSE is to the selection of the initial set (phi parameter in step 5). Next, compare the
other parameters of each demo and take your own criteria about them. Your goal line is at the
new demo to segment the images 5161812.png, birdhouse.png, 271008.jpg, 24077.jpg and
326038.jpg that are in the “ImageSet_Theme3” folder. Then, write a report with your work,
programs and results.
3) In 2013, K. Zhang, L. Zhang, H. Song, and D. Zhang introduced another novel re-initialization
strategy for LS-based image segmentation in a paper published by IEEE Transactions on Image
10 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

Processing entitled “Re-initialization Free Level Set Evolution via Reaction Diffusion”. This
new Reaction Diffusion (RD) is completely free of the costly re-initialization procedure in Level
Set Evolution (LSE), giving rise to the RDLSE algorithm. The RDLSE and other LS-based
image segmentation variants, programmed in Matlab-2017, have been included in the
“Segmentat_LevelSet_RDLSE_Zhang13_Toolbox” folder on the Computer Vision course page
in Moodle. Review the demonstrative, functions and documents in this toolbox. Then run the
six implemented methods. Your programming challenge is to create a new demonstrative with
a way to build the initial set (phi parameter) according to the image you want to segment. To
do this use the same images as in quiz 2) of this assignment. Write a report detailing your work
and showing the results.

Assignment # 11:
1) The Local Binary Patterns (LBP) algorithm is a non-parametric descriptor whose aim is to
efficiently summarize the local structures of images. LBPs were originally designed for texture
description, but are currently a popular technique used for image representation and
segmentation due to their simplicity, high discriminative power and tolerance against
illumination changes. The LBP operator assigns a label to every pixel of an image by
thresholding the 3x3-neighborhood of each surrounding pixel with the center pixel value and a
decimal representation is then obtained from the binary sequence. Some of its variants are Local
Ternary Patterns (LTP), Local Quinary Patterns (LQP), Local Line Binary Pattern (LLBP) and
Local Derivative Pattern (LDP). Graph-based segmentation methods and LBP features have
also been proposed. Then, your written argumentation consists of doing a literature review on
recent (post 2017) developments and applications related to LBP-based image segmentation
methods.
2) In the excellent book Computer Vision Using Local Binary Patterns by M. Pietikäinen, A.
Hadid, G. Zhao, and T. Ahonen, published by Springer in 2011, many practical procedures
related to LBP can be experienced. By the way, one of the algorithms in this book is based on
the work of T. Ojala, M. Pietikäinen, and T. Maenpaa, published in 2002 in IEEE Transactions
on Pattern Analysis and Machine Intelligence, entitled “Multiresolution Gray-Scale and
Rotation Invariant Texture Classification with Local Binary Patterns”, where the authors
presents a theoretically very simple, but efficient, multiresolution approach to gray-scale and
rotation invariant texture classification based on local binary patterns and nonparametric
discrimination of sample and prototype distributions. Based on this method you will find a nice
texture segmentation procedure, programmed in Matlab-2017, in the
“Segmentat_LBP_Pietikainer11_Toolbox” folder that you can download from the Computer
Vision course page on Moodle. Review the demonstrative, functions and documents in this
toolbox. Note that there are several examples in some of the functions. Then run the
demonstrative using the images texture01.jpg, texture02.jpg, texture03.png, texture04.jpg and

11 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

texture05.jpf found in the “ImageSet_Theme3” folder. Try your best to segment each region of
the image according to its textures and number of regions. Then, write a report with your work,
programs and results.
3) Matlab-2017 has the extractLBPFeatures.m function to extract LBP features based on the work
of T. Ojala, M. Pietikainen, and T. Maenpaa entitled "Multiresolution Gray Scale and Rotation
Invariant Texture Classification with Local Binary Patterns" published in 2002 in IEEE
Transactions on Pattern Analysis and Machine Intelligence. Actually, LBP acts as a feature
extractor in segmentation procedures. For example, read the paper by X. Yi and M. Eramian
published in 2016 in IEEE Transactions on Image Processing, Vol. 25, No. 4, pp: 1626-1638,
with the title “LBP-Based Segmentation of Defocus Blur” that you can download from the
“Papers_Theme3” on the Computer Vision course page in Moodle. Your programming
challenge is to implement a segmentation procedure to segment focused regions in short focal
length images. To do this, use the images Focused01.png, Focused02.png, Focused05.png,
Focused21.png, and Focused41.jpg found in the “ImageSet_Theme3” folder. You can follow
the Yi and Eramian paper or implement your own ideas. Write a report explaining your project,
programs (scripts and functions), as well as samples of your results.

Assignment # 12:
1) Mean Shift (MS) clustering is a non-parametric kernel density estimation technique that clusters
a d-dimensional data set based on repeated updating of the mean. MS is one of the most recent
segmentation methods (introduced in 2002), which always searches for the direction of
maximum increase in the density gradient. As a non-parametric and non-supervised iterative
segmentation algorithm, MS do not need of prior knowledge of how many number of clusters
are needed. MS iteratively moves each data point to its local mean until convergence by means
of two steps: first define a kernel of influence for each pixel, and then each pixel is initially
assigned a colocalized mean shift point. All the mean shift points that converge to the same
peak are considered to belong to the same group or segment. Reports indicate success in MS
accuracy for over-segmentation, multiple-segmentation, tracking, and clustering applications.
Examples are the MS-based Edge Detection and Image SegmentatiON (EDISON) software and
the Orfeo toolbox. Nevertheless, instability has also been reported, leading to modified versions
of this algorithm to enforce its stability. Then, your written argumentation consists of doing a
literature review on recent (post 2017) developments and applications related to MS-based
image segmentation methods.
2) D. Comaniciu and P. Meer proposed in 2002 what has been one of the most popular works
related to MS in their paper “Mean Shift: A Robust Approach toward Feature Space Analysis”,
which appeared in IEEE Transactions on Pattern Analysis and Machine Intelligence. Two
demonstratives implementing Comaniciu and Meer’s work in Matlab-2017 have been included
in the “Clustering_MeanShift_Comaniciu02_Toolbox” folder on the Computer Vision course
12 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

page in Moodle. Review the demonstratives and documents in this toolbox. Your programming
walkthroughs include an analysis of these two implementations, explain their differences,
advantages or disadvantages, and how parameter settings can modify the final results. It also
includes the runtime and final mean. To generalize your analysis also process the images
Compos3.png, papsmear2014.bmp, 65019.jpg, 124084.jpg and birdhouse.png that are in the
“ImageSet_Theme3” folder. Then, write a report with your work, programs and results.
3) Your programming challenge is an improvement on quiz 2). First, devise a function for MS-
based clustering that can pass as arguments the input image and parameters, including the
feature (spatial and color) of your choice, and return as output the clustered image and the vector
of means. Then, create a new demo in which you first convert original image from RGB color
space to some luminance-chrominance color space (for example, HSV, HSL, Lab, YUV, etc.),
using the colorspace.m function. Next, the image to be clustered is the converted luminance-
chrominance. Finally, the reverse conversion must be performed to return to the RGB color
space. To evaluate this approach, use the same images as in quiz 2). Write a report detailing
your work and showing the results, as well as comparing with achievements in quiz 2).

Assignment # 13:
1) Hyperspectral image segmentation (HSI) involves difficult processing problems that increase
as the dimensionality of the data increase, which encourages the development of new methods.
In addition, the limited availability of training data, which are difficult to obtain in practice,
restrict the usefulness of deep learning methods. However, several machine learning techniques
are applied to extract relevant information from hyperspectral data sets. Among them, Support
Vector Machines (SVMs) are the most advanced discrimination techniques that can be applied
due to their ability to deal with large input spaces efficiently and to produce sparse solutions.
Also, Multinomial Logistic Regression (MLR) is an alternative approach to deal with ill-posed
problems, which has the advantage of learning the class’s own probability distributions.
Likewise, to work effectively with limited training samples, several methodologies have been
proposed, including feature extraction methods such as Principal Component Analysis (PCA),
Linear Discriminant Analysis (LDA), feature extraction by discriminant analysis, multiple
classifiers, and decision fusion. Your written argumentation consists of doing a literature review
on recent (post 2017) developments and applications related to methods for HIS segmentation.
2) In a paper issued in 2011 by IEEE Transactions on Geoscience and Remote Sensing entitled
“Hyperspectral Image Segmentation using a New Bayesian Approach with Active Learning”,
J. Li J.M. Bioucas-Dias and A. Plaza introduce a new supervised Bayesian segmentation
approach which exploits both spectral and spatial information in the interpretation of remotely
sensed hyperspectral data sets. This method is the basis of an SVM using the Composite Kernel
(CK) method subsequently introduced by Li et al. in 2013. A version of the SVM-CK algorithm
programmed in Matlab-2017 can be found in the “Clustering_SVMCK_HSI_Li13_Toolbox”
13 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

folder on the Computer Vision course page in Moodle. Review the demonstrative, functions
and documents in this toolbox. Also run the demo. Your programming walkthroughs consist of
analyzing the effect of modifying the options of classify_svm_mykernel.m function (in_param
variable in the demo) to test another SVM type or kernel type. Then, write a report with your
work, programs and results.
3) The same authors as in quiz 2), led by J. Li, published in 2012 the work “Spectral-Spatial
Hyperspectral Image Segmentation using Subspace Multinomial Logistic Regression and
Markov Random Fields” in IEEE Transactions on Geoscience and Remote Sensing. This work
proposes a new Bayesian approach to HSI segmentation following two main steps: 1) learning,
where posterior probability distributions are modeled by a Multinomial Logistic Regression
(MLR) combined with a subspace projection method, and 2) segmentation, which infers a class
labels image from a posterior distribution built on the learned subspace classifier and on a Multi-
Level Logistic (MLL) prior on the image of labels. The output of the algorithm is based on a
Maximum A Posteriori (MAP) segmentation process that is computed using an efficient min-
cut-based integer optimization technique. A version of the MLR-MLL algorithm has been
programmed in Matlab-2017 and can be downloaded from the
“Clustering_MLRMLL_HSI_Li12_Toolbox” on the Computer Vision course page in Moodle.
Review the demonstrative, functions and documents in this toolbox. Also run the demonstrative.
Your programming challenge is to investigate the influence of n and nall parameters of the
train_test_random_new.m function, as well as  parameter of GraphGut .m function. Use the
AVIRIS and PraviaU datasets. Write a report detailing your work and showing the results.

Assignment # 14:
1) Image segmentation based on Artificial Neural Nets (ANNs) is a broad topic that includes some
classical structures such as Self-Organizing Map (SOM) and Random Vector Functional Link
(RVFL) networks, raised approaches such as Machine Learning (ML) methods, and advanced
schemes such as Deep Learning (DL) methods. Recent DL methods, and in particular
Convolutional Neural Networks (CNNs), say, VGG (developed by the Visual Geometry Group,
Oxford University), Inception or Residual Nets (ResNets), have shown remarkable results in
recognition tasks. However, the segmentation results of DCNN-based methods are still coarse
and limited, and the accuracy of the image segmentation task is much higher than that of the
semantic image segmentation task. Nevertheless, semantic image segmentation is one of fastest
growing areas in computer vision, with a wide variety of applications, such as video surveillance
and autonomous driving, as it provides the necessary context for actions to be taken based on
pixel level scene understanding. In this context, developments in DL have provided a host of
tools to address this problem with increasing efficiently and accuracy. Then, your written
argumentation consists of doing a literature review on recent (post 2017) developments and
applications related to ANN-based image segmentation methods.

14 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

2) Matlab-2017 functions named nctool.m, nnstart.m, selforgmap.m, train.m, and net.m can
implement ANN-based image segmentation. Your programming walkthroughs is to investigate
and demonstrate the full capabilities of Matlab-2017 for ANN-based image segmentation. To
do so, use the images 24077.jpg, 37073.jpg, 69020.jpg, 106024.jpg, 227092.jpg and
Focused41.jpg that are in the “ImageSet_Theme3” folder. For the first five images compute the
objective segmentation quality (full reference) using the “Segmentat_QualityAssess_Toolbox”.
These images and quality toolbox can be downloaded from the Computer Vision course page
on Moodle. Then, write a report with your work, programs and results.
3) Self-Organizing Maps (SOM) are a type of unsupervised ANN capable of reducing the
dimensionality of data while being able to group together similar samples. Consequently, SOMs
have been successfully used for image color clustering. A Matlab-2017 implementation of SOM
for data clustering can be downloaded from the “Clustering_SOM_Color_Toolbox” folder on
the Computer Vision course page in Moodle. Review the demonstrative, functions and
documents in this toolbox. Next, run the demo. Your programming challenge is to investigate
different setting for the SOM. To do this, you must modify the call to the ClustSOM.m function
by adding new arguments, as well as the call to the selforgmap.m function within ClustSOM.m.
Evaluate this toolbox without and with your new demo and function using the same images as
in quiz 2) of this assignment. Write a report detailing your work and showing the results.

Assignment # 15:
1) Particle Swarm Optimization (PSO) algorithms share many similarities with evolutionary
computation techniques based on computational intelligence, although standard PSO does not
use evolutionary operators such as crossover and mutation. PSO emulates the behavior of insect
swarm, herd animals, flocking birds, and schooling fish, where these swarms forage
collaboratively for food. The global optimum is considered to be the location of food. The PSO
algorithm is conceptually simple, easy to implement, and has demonstrated good performance
on many optimization problems. However, it can easily get trapped in a local optima when
solving complex multimodal problems. In order to improve the performance of PSO, some
variant have been introduced, such as Comprehensive Learning PSO (CLPSO), the Unified PSO
(UPSO), the Fitness-Distance-Ratio-based PSO (FDRPSO), Darwinian PSO (DPSO), or
Fractional-Order Darwinian PSO (FODPSO). Then, your written argumentation consists of
doing a literature review on recent (post 2017) developments and applications related to PSO-
based image segmentation methods.
2) Clustering itself can be posed as an optimization problem, so it can be solved by optimization
algorithms. Color clustering is a common stage in many segmentation problems when first
trying to reduce color diversity, mainly when noise affects the color diversity in the image to
be segmented. A color clustering method, based on the PSO algorithm and programmed in
Matlab-2017, can be downloaded from the “Clustering_PSO_Color_Toolbox” folder on the
15 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

Computer Vision course page in Moodle. Your programming walkthroughs is first a review of
demonstrative, functions and documentation in this toolbox. Afterward, run the demo and
investigate how parameters such as maximum number of iterations and population size
influence the final result when the number of classes is two and wen it is five. To do this, use
the images 65019.jpg, 124084.jpg, 181079.jpg, 288001.jpg, and 388016.jpg found in the
“ImageSet_Theme3” folder. These images can be downloaded from the Computer Vision
course page on Moodle. Then, write a report with your work, programs and results.
3) Your programming challenge is to implement a binarization procedure using as inputs the color
clustered images obtained in the previous quiz 2). Note that the ground truth of the proposed
images is in the same folder. You are free to choose the parameters for color clustering as well
as the method for the binarization stage. Compute the objective segmentation quality (full
reference) for each binarized image using the “Segmentat_QualityAssess_Toolbox” which can
be downloaded from the Computer Vision course page on Moodle. Write a report detailing your
work and showing the results.

Assignment # 16:
1) Random fields is one of the most widely used probabilistic methods in computer vision
problems, such as image segmentation, surface reconstruction and depth inference. RF
segmentation methods include different probabilistic models, such as Markov Random Fields
(MRF), a powerful stochastic tool that models joint probability distribution of image pixels,
widely used in computer vision tasks due to its increased resistance to noise. Even so,
conventional MRF has a weakness: the need to explicitly model the probability distribution and
assume that the observed image data are conditionally independent, given the labels. Solutions
have been proposed like the Conditional Random Fields (CRF), an undirected graphical models
for structured prediction, which are trained to maximize the conditional probability of the
outputs given the inputs. Also, Gaussian-based Hidden Markov RF (HMRF), with the
advantage, compared with MRF, that the parameter set is learned in an unsupervised manner.
Moreover, in a HMRF image segmentation problem, there is no training stage, and no prior
knowledge about the foreground/background intensity distribution is needed. Another solution
is the Pairwise Markov Fields (PMF) model that allows unsupervised Bayesian modeling and
segmentation of blurred images. Then, your written argumentation consists of doing a literature
review on recent (post 2017) developments and applications related to random fields-based
image segmentation methods.
2) In the work by A. Sengur, I. Turkoglu and C. Ince “Unsupervised Image Segmentation Using
Markov Random Fields”, presented at the Turkish Symposium on Artificial Intelligence and
Neural Networks (TAINN) 2006, they performed unsupervised segmentation of gray-level
images based on the Markov Random Fields (MRF) model. Their approach used the
Expectation Maximization (EM) algorithm to estimate the input image distribution, and the
16 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

number of the components was automatically determined by the Minimum Message Length
(MML) algorithm. Finally, the segmentation was performed using the Iterated Conditional
Modes (ICM) algorithm. You can download the method of Sengur et al. implemented in Matlab-
2017 from the “Clustering_MRF_MMLICM_Sengur06_Toolbox” folder on the Computer
Vision course page in Moodle. Review the demonstrative, functions and document in this
toolbox. Your programming walkthroughs start running the demo by trying to do the best you
can with the two- and five-classes color clustering of the images 69020.jpg, 86016.jpg,
124084.jpg, 189080.jpg and 208001.jpg that are in the “ImageSet_Theme3” folder. As result,
you will be able to explain the relationship between “potent” and “maxItr” parameters with the
results achieved. Then, write a report with your work, programs and results.
3) In 2012 Q. Wang wrote the following two works: “GMM-Based Hidden Markov Random Field
for Color Image and 3D Volume Segmentation” and “HMRF-EM-image: Implementation of
the Hidden Markov Random Field Model and its Expectation-Maximization Algorithm”. The
code of these two methods programmed in Matlab-2017 have been included in the
“Clustering_GMM_HMRF_Wang12_Toolbox” folder on the Computer Vision course page in
Moodle. Review the demonstrative, functions and document in this toolbox and run both demos.
Your programming challenge is, first of all, to put both toolboxes together into one toolbox with
a single demonstrative. Provide options to the user (such as a menu, etc.), including
visualization of results. Then, the second effort is to assess the segmentation quality of the
binarized images (two classes) using the same images as in quiz 2) and both clustering methods
with the “Segmentat_QualityAssess_Toolbox” which can be downloaded from the Computer
Vision course page on Moodle. Write a report detailing your work and showing the results.

Assignment # 17:
1) The Random Forest is a popular probabilistic tool used in classification and regression tree
analysis. This general tool can be successfully applied to many other problems in computer
vision tasks, like edge detection, semantic segmentation, or 3D location. For the random forest,
the process consist of two parts: train and then detect. Two of the main advantages of random
forest are: speed and minimal storage. Some variants are Bayesian Forests (BF), Canonical
Correlation Forests (CCF), Decision Forest (DF), and Decision Jungles (DJ). Additionally,
based on computational intelligence, Deep Neural Decision Forests (DNDF), or Relating
Cascaded Random Forests to Deep Convolutional Neural Networks (RCRF-DCNN) can be
mentioned. Then, your written argumentation consists of making a literature review on recent
(post 2017) developments and applications related to random forest-based image segmentation
methods.
2) Edge detection has remained a fundamental task in computer vision because its accurate
detection is a critical preprocessing step for a variety of tasks, including object recognition or
segmentation. Based on the theory of Random Forest, in 2015 P. Dollár and C.L. Zitnick
17 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

published the paper “Fast edge detection using structured forests” in IEEE Transactions on
Pattern Analysis and Machine Intelligence. These authors termed their method as Structured
Edge (SE) and a version of it can be downloaded from the “Edges_SE_Dollar15_Toolbox”
folder on the Computer Vision course page in Moodle. Review the demonstrative, functions
and documents in this toolbox and run the demo named Demo_SE_Dollar15.m. Your
programming walkthroughs is to investigate how the model detection options (number of scales,
sharpness, number of trees, number of threads and non-maximum suppression) influence the
final result. To do this see the function edgesTrain.m. You can also experiment by modifying
another options. The images to use are 124084.jpg, 181079.jpg, 153093.jpg, 227092.jpg and
376043.jpg which you can find in the “ImageSet_Theme3” folder on the Computer Vision
course page in Moodle. Then, write a report with your work, programs and results.
3) In the same toolbox used for quiz 2) you will find another demo termed
Demo_SE_Superpixels.m. Your programming challenge comprise first upgrading this demo
until it becomes equivalent to Demo_SE_Dollar15.m. After that, evaluate the upgraded demo
as you do in quiz 2). Write a report detailing your work and showing the results.

Assignment # 18:
1) The Random Walk (RW) algorithm is one of the most influential interactive (supervised)
segmentation methods. Interactivity in RW means supervision, that is, the user labels some
image pixels (seeds) and the algorithm assumes that a random walker starts from each unlabeled
pixel and calculates the probability that these random walkers reach the already labeled pixels
first. At each pixel, the label with the greatest probability is chosen as the final label, which
produces the final interactive segmentation result. A limiting property of the RW algorithm is
that each segment has to be connected to a seed. Consequently, several variants have been
proposed such as the Random Walker with Restart (RWR), introduced to handle natural images
with complex textures. Another variant is the “lazy” RW (LRW), proposed for superpixel
segmentation. An additional variant is the Partially Absorbing RW (PARW), used for video. A
further variant is the sub-Markov random walk (subRW) approach, which can be considered as
an extension of LRW, RWR and PARW in which additional nodes are added in the base
(original) image graph providing different types of image information. More recently,
normalized versions like NRW, NLRW and NsubRW have also been proposed. All of them
with their own properties and limitations. Then, your written argumentation consists of doing a
literature review on recent (post 2017) developments and applications related to RW-based
image segmentation methods.
2) Based on a Random Walk (RW) algorithm, in this quiz you will learn how to interactively
segment a complex image with more than one region-of-interest (ROI) in its foreground. One
of the classic RW-based image segmentation algorithm was in published in 2010 by S.
Andrews, G. Hamarneh, and A. Saad in their work “Fast Random Walker with Priors using
18 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

Precomputation for Interactive Medical Image Segmentation” included in Lecture Notes in

Computer Science, Medical Image Computing and Computer-Assisted Intervention (MICCAI).
A version of the Fast Random Walker with Priors (FRWP) can be downloaded from the
“Segmentat_FRWP_Andrews10_Toolbox” folder on the Computer Vision course page in
Moodle. However, the original code can be downloaded from http://fastrw.cs.sfu.ca/. Review
the demonstrative, functions and documents in this toolbox. Carefully examine seeded (labeled)
images in the “image” folder. Next, run the demo using the first default images in menu,
parameters and non-interactive seeding option to familiarize yourself with the code when an
image has been previously seeded. After that, make your own seeds. Note that you can choose
whether or not to save the last seeded image. Now, you are able to do your best by trying to
segment, without tuning parameters, the images 24077.jpg, 65019 .jpg, 209070 .jpg, 271008.jpg
and birdhouse.png that you can find in the “ImageSet_Theme3” folder on the Computer Vision
course page in Moodle. Remember that if the target to be segmented is not homogeneous,
containing much diversified textures, the use of only a small number of seed points does not
usually provide enough information of the target, and therefore a satisfactory result cannot be
obtained. That is, as an interactive method, the performance depends on the subjectivity of the
seeding stage. Then, write a report with your work, programs and results.
3) Your programming challenge is to investigate the setting of the main parameters, such as
“precomp” and “upriors”, in “Segmentat_FRWP_Andrews10_Toolbox”. Choose one of the
images proposed in quiz 2). Write a report detailing your work and showing the results.

Assignment # 19:
1) Region Growing (RG) is a traditional interactive (supervised) image segmentation method
based on hierarchical region aggregation using local similarity rules. This mean that RG is a
region-based technique in which the user first labels some image pixels (seeds) and then all its
remaining neighbor pixels are compared with it according to the manually selected threshold T,
as in the classical Seeded Region Growing (SRG). RG is a fast, effective and strong image
segmentation algorithm. However, the initial seed selection by trial and error may not always
provide the expected results, the results may be erroneous if a seed point falls on a noisy point
or the user makes an error in the choice of seed points. Additionally, different seed points and
threshold values create different segmentation results. So, some variants have been introduced
in the recent past, such as best merge region growing and its variants. Then, your written
argumentation consists of doing a literature review on recent (post 2017) developments and
applications related to RG-based image segmentation methods.
2) Color clustering is one of the most successful approaches for image segmentation due to the
highest dimensionality of features in the three-dimension of color space versus the single
dimension of grayscale images. As in human vision, color information helps to discriminate
details. Even so, the weakness of color clustering has been to ignore the spatial relationship
19 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

between image pixels which commonly produce regions with artificially generated edges
known as staircase artefacts. In the paper “Unsupervised color image segmentation using a
dynamic color gradient thresholding algorithm” published in 2008 by G.P. Balasubramanian et
al., they perform the segmentation with emphasis on the use of homogenous color regions and
color transitions without generating edges. Using color gradient they aids the RG process rather
than generating edges. Also, to ensure consistency of the segmentation with the image regions,
RG is followed by a similarity measure-based region-merging (RM) step. A version of the
unsupervised region growing and region merging (URGM) algorithm of Balasubramanian et al.
programmed in Matlab-2017 has been placed into the
“Clustering_URGM_Balasub08_Toolbox” folder on the Computer Vision course page in
Moodle. Review the demonstrative, functions and document in this toolbox. Next, run the demo
using the preset parameters. Your programming walkthroughs is to first make your best color
clustering of the images 21077.jpg, 5161812.png, 111876273311.png, birdhouse.png and
twohorses.png that you can find in the “ImageSet_Theme3” folder on the Computer Vision
course page in Moodle. Next, attempt to use some other color space (for example HSV, HSL,
Lab, etc.) to perform the clustering using colorspace.m function. Then, write a report with your
work, programs and results.
3) A simple way to implement a RG-based color clustering is using the Matlab function termed
detectMSERFeatures.m. Therefore, your programming challenge is to make a toolbox for color
clustering using this function. We recommend you use the following parameters setting to get
started: ThresholdDelta = 2.5, RegionAreaRange = [140, 8000], MaxAreaVariation = 0.2. To
test your toolbox use the same images than in quiz 2) of this assignment. Write a report detailing
your work and showing the results.

Assignment # 20:
1) Saliency-based segmentation is a computer vision approach in which the image is divided into
separate compact foreground and background, emulating the visual selective attention
mechanism of human visual system. That is, the salient region is distinguished from its
neighbor. Salient object detection or salient object segmentation is commonly interpreted in
computer vision as a process involving two stages: 1) detecting the most salient object and 2)
segmenting the precise region of that object. Abundant ideas have been proposed to address the
saliency-based segmentation. These include the Saliency Driven Region-Edge based Level set
(SDREL) model, Saliency Prediction with Generative Adversarial Networks (SalGAN),
Saliency Model Implementation Library for Experimental Research (SMILER), Pixel-wise
Contextual Attention Neural Net (PiCANet), Depth-induced Multi-scale Recurrent Attention
Network (DMRANet), and much others. Then, your written argumentation consists of doing a
literature review on recent (post 2017) developments and applications related to saliency-based
image segmentation methods.

20 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

2) In this quiz you will learn about an interesting way to construct visual saliency maps. Saccadic
eye-movement is an important mechanism of the human visual system that allow us to
efficiently explore complex environment guided by the attention caused by salient objects. A
remarkable work by X. Sun, H. Yao, R. Ji, and X.-M. Liu presented in 2014 a unified statistical
framework for modeling both Saccadic eye movements and visual saliency with the title
“Toward statistical modeling of saccadic eye-movement and visual saliency”, published in
IEEE Transactions on Image Processing. A version of this approach, known as the Super
Gaussian component Pursuit (SGP) model, as well as an implementation based on Saccadic
eye-movement, have been programed in Matlab-2017 in the “Saliency_SGP_Sun14_Toolbox”
folder on the Computer Vision course page in Moodle. Review the two demos (first the
Saccade-based script), run them, as well as review functions and documents. Next your
programming walkthroughs is to do your best to get the saliency maps of the images
Focused01.png, Focused02.png, Focused05.png, Focused21.png and Focused41.jpg that you
can find in the “ImageSet_Theme3” folder on the Computer Vision course page in Moodle.
Then, write a report with your work, programs and results.
3) Segmentation of focused regions in low-depth-of-field (LDF) images is also a common task in
computer vision systems. In this quiz you will work with two methods for saliency mapping in
LDF images and your programming challenge is to combine both methods in a unified approach
programmed in Matlab-2017 that take advantages of each method. These methods are in
“Saliency_LDF_Toolbox” and “Saliency_PQFT_Guo08_Toolbox” folder. To do this, first
review both toolboxes and run demos with preset parameters. Put your code in a new toolbox
including demo, functions, documents, readme, etc. When you reach your goal evaluate the new
toolbox using the same images than in quiz 2). Write a report detailing your work and showing
the results.

Assignment # 21:
1) Superpixel-based image segmentation (SP) methods are perhaps one of the most profuse
methods for image segmentation and several SP approaches have been published in recent
years. Actually, SP is an over-segmentation method because it is easier to merge superpixels in
post stages than to split them. Wang et al. (2017), classify SP methods simply into graph-based
and clustering-based considering the segmentation model. However, according to D. Stutz, A.
Hermans and B. Leibe (2017), SP methods can be classified into watershed-based, density-
based, graph-based, contour evolution, path-based, clustering based, energy optimization, and
wavelet-based methods. Latterly, Xie et al. (2019) divided SP-based image segmentation
algorithms into unsupervised and supervised methods. More recently, A. Ibrahim and E.-S.M
El-Kenawy (2020) categorized SP algorithms into graph-based methods and gradient-ascent
based approaches. Then, your written argumentation consists of making a literature review on

21 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

recent (post 2017) developments and applications related to SP-based image segmentation
methods and propose your own categorization.
2) The Matlab-2017 functions named superpixels.m and superpixels3.m implement 2-D and 3-D
image superpixel oversegmentation using the Simple Linear Iterative Clustering (SLIC) method
published in 2012 by R. Achanta, et al., in the paper entitled “SLIC Superpixels Compared to
State-of-the-art Superpixel Methods” in IEEE Transactions on Pattern Analysis and Machine
Intelligence. Your programming walkthroughs is to investigate and demonstrate the full
capabilities of Matlab-2017 for SP-based image segmentation. For this, use the images
24077.jpg, 37073.jpg, 69020.jpg, 106024.jpg, 227092.jpg and Focused41.jpg found in the
“ImageSet_Theme3” folder. For the five images compute the objective segmentation quality
(full reference) using the “Segmentat_QualityAssess_Toolbox”. These images and quality
toolbox can be downloaded from the Computer Vision course page on Moodle. Then, write a
report with your work, programs and results.
3) In a 2018 work R. Giraud, V.-T. Ta, and N. Papadakis, issued in Computer Vision and Image
Understanding, with the title “Robust superpixels using color and contour features along linear
path” introduced the Superpixels with Contour Adherence using Linear Path (SCALP)
algorithm. You can download a version of SCALP algorithm programmed in Matlab-2017 from
the “Superpixel_SCALP_Giraud18_Toolbox” folder in the Computer Vision course page in
Moodle. Review the demo and documentation, and then run the demo with the preset
parameters. As you can see SCALP can run with and without contour prior. Next do your best
superpixel segmention on the same images as in quiz 2) without contour prior. Therefore, your
programming challenge is to replace the Canny edge segmentation method for contour prior by
one that outperform it. You are free to choose the method, however there are some edge
segmentation toolboxes on the Computer Vision course page in Moodle. Write a report detailing
your work and showing the results.

Assignment # 22:
1) Image partitioning based on the Potts Model (PM) is an important and challenging basic task in
image processing. PM is an image segmentation strategy in which the goal is to group parts of
the image with similar characteristics, such as colors or textures, in order to extract essential
information from the image. The image partitioning problem is usually formulated as a
minimization problem for a certain cost functional, where the regularity term is the total length
of the partition boundaries, leading to the classical PM, also known as the piecewise constant
Mumford-Shah model. However, the basic Potts model, although it performs well in many
cases, may not always capture the image characteristics well enough. Recent improved versions
have been reported that Potts’ minimization strategies give reasonable segmentation results
compared to some other state-of-the art methods. Then, your written argumentation consists of

22 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

doing a literature review on recent (post 2017) developments and applications related to PM-
based image segmentation methods.
2) The PM-based image color clustering approach proposed in 2015 by M. Storath, A. Weinmann,
J. Frikel, and M. Unser in the work “Joint Image Reconstruction and Segmentation Using the
Potts Model”, can be downloaded from the “Clustering_PottsLab_Storath15_Toolbox” folder
on the Computer Vision course page in Moodle. First, read the Radme.txt file and do what it
says. Your programming walkthroughs is to do your best colors clustering of the images
21077.jpg, 37073.jpg, 153077.jpg, 181079.jpg, llama.bmp that are in the “ImageSet_Theme3”
folder. Then, write a report with your work, programs and results.
3) One of the strengths of the Pottts Model is its robustness to noise. Therefore, your programming
challenge is to evaluate the PottsLab toolbox of Storath (2015) when the images are corrupted
with additive white Gaussian noise (AWGN) with different signal-to-noise ratio (SNR) values.
Use the Matlab function named imnoise.m with the 'gaussian ' option of zero mean and standard
deviation () from 0.05 to 0.5. For each value of noise , generalize your evaluation by
averaging the SNR increase and Potts energy reduction for the five images listed in quiz 2).
Write a report detailing your work and showing the results.

Assignment # 23:
1) Thresholding techniques (based on threshold selection), which can be though as pixel
classification, are perhaps one of the simplest, undemanding, robust in the presence of noise,
and successful methods for segmenting grayscale images into non-semantic categories,
especially in applications where speed is an important factor. Image thresholding deals with
subdividing the image into regions of meaningful information based on a simple decision. The
reason for classifying pixels by their grey level is that pixels with similar level in a nearby
region usually belong to the same object. Then, thresholding as a segmentation method belongs
to a class that assumes that image pixels can be classified into different groups according to
gray level through various thresholds. However, most advanced methods incorporate the spatial
information between pixels into the thresholding process. These methods, known as two-
dimensional thresholding, constitute the state-of-the-art in image thresholding. Then, your
written argumentation consists of doing a literature review on recent (post 2017) developments
and applications related to thresholding-based image segmentation methods.
2) Image thresholding can be performed in Matlab-2017 using some functions such as
imbinarize.m, graythresh.m, otsuthresh.m, multithresh.m and adaptthresh.m. Actually,
imbinarize.m does not compute the threshold value, while graythresh.m, multithresh.m and
otsuthresh.m use Otsu’s method published in 1979 in IEEE Transactions on Systems, Man, and
Cybernetics, with the title “A Threshold Selection Method from Gray-Level Histograms”. Then,
adaptthresh.m computes a locally adaptive threshold that can be used with the imbinarize.m

23 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

function to convert an intensity image to a binary image. Your programming walkthroughs is

to investigate and demonstrate the full capabilities of Matlab-2017 for thresholding-based
image segmentation into two (bi-class) and multiple classes, say five classes. For this, use the
images 24077.jpg, 37073.jpg, 69020.jpg, 106024.jpg, 227092.jpg and Focused41.jpg that are
in the “ImageSet_Theme3” folder. For the first five images compute the objective (full
reference) binarization quality using the “Segmentat_QualityAssess_Toolbox”. These images
and quality toolbox can be downloaded from the Computer Vision course page on Moodle.
Then, write a report with your work, programs and results.
3) Otsu’s method for global image thresholding is successful when the image histogram is
bimodal, with clear Gaussian distributions of intensities in the image between two classes,
foreground and background. However, the histogram of natural scenes can also be monomodal
o multimodal, which require other thresholding methods. Moreover, there are other thresholding
methods specially designed for multilevel or adaptive thresholding. Some of these thresholding
methods have been included in the “Thresholding_Toolbox” where approximately forty
methods with their documentation and demonstratives can be found. Review the demos,
function help and documents. Next run the three demos, which show the usage off all global,
local and multilevel thresholding methods in this toolbox. After that, try your best to binarize
the same images as in quiz 2) of this assignment and assessing the objective binarization quality.
Write a report detailing your work and showing the results.

Assignment # 24:
1) Total Variation (TV) is a regularization functional for imaging problems. Because it imposes
an L1 penalty on differences between adjacent pixels, it tends to result in images with piecewise
constant regions. TV minimization appears in many clustering and segmentation problems.
Numerical TV minimization has long been challenging, but recent advances in large-scale
nonsmooth convex optimization, with efficient primal-dual splitting schemes and alternating
directions methods, have made the implementation of TV minimization relatively easy and
efficient. Then, your written argumentation consists of doing a literature review on recent (post
2017) developments and applications related to TV-based image segmentation methods.
2) In 2017, L. Condat presented in SIAM Journal of Imaging Science a remarkable paper
introducing a new definition of total variation (TV), which produces sharper edges and better
isotropy than the classical definition, under the title "Discrete Total Variation: New Definition
and Minimization". After a review of the classical definitions of discrete TV and its properties,
Condat introduced four new formulations for anisotropic TV, isotropic TV, upwind TV and a
proposed new variant of discrete TV. These five TV formulations (DTV, ATV, ITV, UTV and
DTV-V) have been programmed in Matlab-2017 and can be downloaded from the
"Clustering_TV_Condat17_Toolbox" folder on the Computer Vision course page in Moodle.
Review the demonstratives, functions and documentation. Next run each demo to do your best
24 
 
 Computer Vision Assignments. Theme 3: Image Segmentation

colors clustering on five classes of the images 65019.jpg, 153093.jpg, 181079.jpg, 376043.jpg
and birdhouse.png that are in the “ImageSet_Theme3” folder. Your programming walkthroughs
consist of creating five new demos in which the image is contaminated with additive white
Gaussian noise (AWGN) before color clustering. For this you can use the Matlab-2017 function
named imnoise.m with zero mean and standard deviation () from 0.01 to 0.1 to perceptually
compare the quality of the clustering with and without noise contamination. Then, write a report
with your work, programs and results.
3) As you have seen, demonstratives in the “Clustering_TV_Condat17_Toolbox” folder run only
for five classes clustering. However, it is common to cluster color in other number of classes
than five. Then, your programming challenge is to create another five demos to make each TV
method in the Condat’s toolbox run for any arbitrary number of classes greater than or equal to
two. Write a report detailing your work and showing the results.

25