Journal of Microscopy, Vol. 198, Pt 1, April 2000, pp. 63–75.

Received 31 March 1999; accepted 28 October 1999

The influence of the regularization parameter and the

first estimate on the performance of Tikhonov regularized
non-linear image restoration algorithms

G. M. P. VAN KEMPEN & L. J. VAN VLIET

Pattern Recognition Group, Faculty of Applied Physics, Delft University of Technology,
Lorentzweg 1, 2628 CJ Delft, The Netherlands

Key words. First estimate, image restoration, iterative restoration algorithms,

regularization parameter.

Summary 1. Introduction
This paper reports studies on the influence of the The goal of image restoration is to ‘undo’ the blurring and
regularization parameter and the first estimate on the noise artefacts that are imposed on the image during image
performance of iterative image restoration algorithms. We acquisition. In other words, image restoration algorithms
discuss regularization parameter estimation methods that try to restore the ‘original’ image from the acquired image
have been developed for the linear Tikhonov–Miller filter to by deconvolving the blurring imposed by the point spread
restore images distorted by additive Gaussian noise. We function (PSF) and by reducing the noise imposed by the
have performed experiments on synthetic data to show that image recording.
these methods can be used to determine the regularization The performance of non-linear iterative image restora-
parameter of non-linear iterative image restoration algo- tion algorithms is highly dependent on input parameters of
rithms, which we use to restore images contaminated by these algorithms, such as the PSF, background value,
Poisson noise. We conclude that the generalized cross- regularization parameter, first estimate and stop criterion.
validation method is an efficient method to determine a For example, the dependence of the performance of these
value of the regularization parameter close to the optimal algorithms on the estimation of the PSF and the back-
value. We have also derived a method to estimate the ground estimation have been shown before (van der Voort
regularization parameter of a Tikhonov regularized version & Strasters, 1995; van Kempen, 1999; van Kempen et al.,
of the Richardson–Lucy algorithm. 1997). We studied the performance of non-linear iterative
These iterative image restoration algorithms need a first regularized image restoration algorithms as a function of
estimate to start their iteration. An obvious and frequently the regularization parameter and the first estimate. The first
used choice for the first estimate is the acquired image. estimate is a first estimate of the ‘original image’ that
However, the restoration algorithm could be sensitive to iterative restoration algorithms require as input.
the noise present in this image, which may hamper the We studied regularized image restoration algorithms like
convergence of the algorithm. We have therefore compared the iterative constrained Tikhonov–Miller algorithm (ICTM)
various choices of first estimates and tested the convergence (Lagendijk & Biemond, 1991; van der Voort & Strasters,
of various iterative restoration algorithms. We found that 1995), the Carrington algorithm (Carrington, 1990;
most algorithms converged for most choices, but that Carrington et al., 1995) and the Tikhonov regularized
smoothed first estimates resulted in a faster convergence. Richardson–Lucy algorithm, which we refer to as the RL–
Conchello algorithm (Conchello & McNally, 1996). These
Correspondence: G. M. P. van Kempen, Central Analytical Sciences, Spectroscopy
algorithms balance the fit of their restoration result to the
& Microscopy Unit, Unilever Research Vlaardingen, Olivier van Noortlaan 120,
acquired image with an a priori model of the restoration
3133 AT Vlaardingen, The Netherlands.
1
Although mathematically equivalent to the maximum a priori (MAP) approach
result1 (Andrews & Hunt, 1977). This balance is deter-
used here, the maximum penalized likelihood method (MPL) employs the regular- mined by the regularization parameter, which determines to
ization term as a means to ameliorate some undesired behaviour of the solution what extent the restoration result is governed by the noisy
instead of an, often hard to obtain, a priori probability distribution of the scene recorded data or by the prior. A large value for the
under observation. regularization parameter results in a stronger influence of

q 2000 The Royal Microscopical Society 63

64 G . M . P. VA N KEMPEN A ND L. J. VA N VLIET

Fig. 1. Restoration of a sphere using the

ICTM algorithm for a high value (0.1) (left),
a low value (0.00001) (right) and a well-
chosen value (0.006) for the regularization
parameter.

the regularization prior on the restoration result, yielding in function of these regularization parameter estimations. In
general a smooth result, whereas a low value of the section 5, we derive a method to estimate the regularization
regularization parameter will make the restoration algo- parameter for the RL–Conchello algorithm. Section 6
rithms more sensitive to the noise in the acquired image (see discusses various first estimates and shows how the
Fig. 1). The regularization parameter therefore is an convergence of various iterative restoration algorithms
important factor in the performance of these image depends on the choice of first estimate.
restoration algorithms. We investigate the influence of the
regularization parameter on the performance of various
restoration algorithms and compare various methods for 2. Image restoration
determining the regularization parameter for the ICTM and
Carrington algorithm. Furthermore, we show how the 2.1. Classical image restoration: the Tikhonov–Miller
regularization parameter for the RL–Conchello algorithm restoration filter
can be estimated. We assume that the image formation in a confocal
This paper presents a more thorough investigation of fluorescence microscope can be modelled as a linear
the influence of the regularization parameter of the translation-invariant system distorted by noise
performance of non-linear restoration algorithms than
mðx; y; zÞ ¼ N ðhðx; y; zÞ ⊗ f ðx; y; zÞ þ b ðx; y; zÞÞ ð1Þ
previous work (van Kempen et al., 1997). It compares a
computed regularization parameter with the value that is In this equation f represents the input signal, h the point
optimal in mean square error sense (a value that can only spread function, b a background signal (which we assume
be obtained in a simulation experiment). Furthermore, to be constant), N a general noise distortion function, and m
this paper presents a novel method for the estimation the recorded fluorescence image. For scientific-grade light
of the regularization parameter for the RL–Conchello detectors N is dominated by Poisson noise.
algorithm. In classical image restoration, the signal-dependent
The algorithms mentioned are iterative, non-linear image Poisson noise is approximated by additive Gaussian noise.
restoration algorithms. These iterative algorithms need a Using this additive Gaussian noise model for N we rewrite
first estimate to start their iterations. A commonly used (1) as
choice is the acquired image itself. Although this image is gðx; y; zÞ ¼ m ðx; y; zÞ ¹ b ðx; y; zÞ ¼ hðx; y; zÞ ⊗ f ðx; y; zÞ
often close enough to the end result to guarantee a
convergence of the iteration, the acquired image contains þ nðx; y; zÞ ð2Þ
noise, to which the algorithms can be sensitive. Therefore, After sampling Eq. (2) becomes
it is useful to compare this choice of first estimate to
Mx X
X My X
Mz
alternatives. We have investigated smoothed versions of the
g½x; y; zÿ ¼ h½x ¹ i; y ¹ j; z ¹ kÿ f ½i; j; kÿ
acquired image as first estimate as well as the result of
i¼1 j¼1 k¼1
linear Tikhonov restoration. This latter choice will in
general be much closer to the final restoration result, þ n½x; y; zÿ ð3Þ
especially for confocal imaging, therefore promising (much) with Mx, My and Mz the number of sampling points in the x,
faster convergence. y and z dimension, respectively. For convenience we will
In the next section we formulate a mathematical adopt a matrix notation
description of the image formation in a confocal fluores-
g ¼ Hf þ n ð4Þ
cence microscope and use it to discuss linear and non-linear
image restoration techniques. In section 3, we review where the vectors f, g and n of length M (M ¼
various methods to estimate the regularization parameter of Mx þ My þ Mz) denote the object, its image and the additive
the Tikhonov functional. The next section compares the Gaussian noise, respectively. The MxM matrix H is the
performances of the ICTM and Carrington algorithms as blurring matrix representing the PSF of the microscope.

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 S TA RT I N G A N D R E G U L A R I Z I N G I M AG E R E S TO R AT IO N 65

The Tikhonov–Miller filter, a classical image restoration ICTM algorithm (Lagendijk & Biemond, 1991; van der Voort
filter, is a convolution filter operating on the measured & Strasters, 1995) finds the minimum of Eq. (7) using the
image. It can be written as method of conjugate gradients (Press et al., 1992). Setting the
negative intensities to zero after each iteration implements
f̂ ¼ Wg ð5Þ
the non-negativity constraint.
ˆ
with W the linear restoration filter and f its result. The
Tikhonov–Miller filter is derived from a least squares 2.2.2. The Carrington algorithm. Like the ICTM algorithm,
approach, which is based on minimizing the squared the Carrington algorithm (Carrington, 1990; Carrington
difference between the acquired image and a blurred et al., 1995) minimizes the Tikhonov functional under the
estimate of the original object, constraint of non-negativity. However, the algorithm is based
on a more solid mathematical foundation. The derivation of
kHf̂ ¹ gk2 ð6Þ
the Carrington algorithm as given in this section is based on
However, a direct minimization of Eq. (6) will produce Verveer (personal communication, 1995).
undesired results, as it does not take into account the (high) Given the Tikhonov functional Eq. (7) (without the
frequency components of fˆ that are set to zero by the regularization matrix) and its derivative
convolution with H. 1 T
Finding an estimate fˆ from Eq. (6) is known as an ill- = Fðf̂Þ ¼ H ðHf̂ ¹ gÞ þ lf̂ ð10Þ
2 f̂
posed problem (Tikhonov & Arsenin, 1977). To address this
issue Tikhonov defined the regularized solution fˆ of Eq. (4) we want to find the non-negative solution of fˆ that
the one that minimizes the well-known Tikhonov functional minimizes this functional. Then the Kuhn–Tucker conditions
(Tikhonov & Arsenin, 1977) apply (Carrington, 1990):

Fðf̂Þ ¼ kHf̂ ¹ gk2 þ lkRf̂k2 ð7Þ =f̂ Fi ¼ 0 and f̂ i > 0 or =f̂ Fi $ 0 and f̂ i ¼ 0 ð11Þ

with k·k2 the Euclidean norm. In image restoration l is Then from Eq. (10) we find
known as the regularization parameter and R as the 1 T
=f̂ F ¼ 0 → f̂ ¼ H ðg ¹ Hf̂Þ ; HT c ð12Þ
regularization matrix. The Tikhonov functional consists of l
a mean square error fitting criterion and a stabilizing From these conditions we see that:
energy bound which penalizes solutions of fˆ that oscillate
wildly due to spectral components which are dominated by f̂ ¼ HT c where HT c > 0 and f̂ ¼ 0 where HT c # 0 ð13Þ
noise. The minimum of F can be found by solving This can be written as
=f̂ Fðf̂Þ ¼ 2H ðHf̂ ¹ gÞþ2lR Rf̂ ¼ 0
T T
ð8Þ f̂ ¼ maxð0; HT cÞ ¼ PðHT cÞ ð14Þ
which yields the well-known Tikhonov–Miller (TM) solution On the set where H c > 0 we then obtain after inserting (14)
T

WTM (Miller, 1970; Tikhonov & Arsenin, 1977) into (8) that
WTM ¼ ðHT H þ lRT RÞ¹1 HT ð9Þ =f̂ F ¼ HT ðHPðHT cÞ ¹ g þ lcÞ ¼ 0 ð15Þ
The linear nature of the Tikhonov–Miller makes it This is equivalent to minimizing the functional W (on the set
incapable of restoring frequencies for which the PSF has a HTc > 0)
zero response. Furthermore, linear methods cannot restrict
1 1
the domain in which the solution should be found. This WðcÞ ¼ kPðHT ck2 ¹ cT g þ lkck2 ð16Þ
property is a major drawback as the intensity of an imaged 2 2
object represents light energy, which is non-negative. Since W is strictly convex and twice continuously differenti-
The ICTM algorithm, the Carrington algorithm and the able, a conjugate gradient algorithm can be used to
Richardson–Lucy algorithm are frequently used in fluores- minimize W.
cence microscopy (Holmes, 1988; Carrington, 1990; van
der Voort & Strasters, 1995; Conchello & McNally, 1996;
van Kempen et al., 1997; van Kempen, 1999; Verveer et al.,
2.3. Maximum likelihood restoration: the Richardson–Lucy
1999). These iterative, non-linear algorithms tackle the
algorithm
above mentioned problems in exchange for a considerable By contrast with the two algorithms discussed previously
increase in the computational complexity. the Richardson–Lucy is not derived from the image
formation model (4) which assumes additive Gaussian
2.2. Constrained Tikhonov restoration noise. Instead the general noise distortion function N is
assumed to be dominated by Poisson noise.
2.2.1. The iterative constrained Tikhonov–Miller algorithm. The A fluorescence object can be modelled as a spatially

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

66 G . M . P. VA N KEMPEN A ND L. J. VA N VLIET

inhomogeneous Poisson process F with an intensity proposed to estimate the regularization parameter for the
function f (Snyder & Miller, 1991), linear Tikhonov–Miller filter. In the following section we
F ¹f i will test whether these methods are suitable to be applied to
f i ie
PðFi jf i Þ ¼ ð17Þ non-linear restoration algorithms for the restoration of
Fi ! Poisson distorted images.
The image formation of such an object by a fluorescence
microscope can be modelled as a translated Poisson process
3.1. The SNR method
(Snyder & Miller, 1991). This process models the transfor-
mation of F into a Poisson process m subjected to a The signal-to-noise ratio (SNR) method sets l equal to the
conditional probability density function H, inverse of the signal-to-noise ratio
E½mÿ ¼ Hf þ b ð18Þ e e
lSNR ¼ ¼ P ð22Þ
E jg ¹ bj2
with b the mean of an independent (background) Poisson
process. The log likelihood function of such a Poisson with e the total power of the noise and E the total power of
process is given by (Snyder & Miller, 1991) the object in the image.
X
LðfÞ ¼ ¹ Hf þ mT lnðHf þ bÞ ð19Þ
3.2. The method of constrained least squares
where we have dropped all terms that are not dependent on
f. The maximum of the likelihood function L can be found In Galatsanos & Katsaggelos (1992) the methods of
iteratively using the EM algorithm as described by Dempster constrained least squares, generalized cross-validation and
et al. (1977). This iterative algorithm was first used by Vardi et maximum likelihood are described to determine l for the
al. (1985) in emission tomography. Holmes (1988) introduced Tikhonov–Miller algorithm. These methods define different
the algorithm to microscopy. Applying the EM algorithm to Eq. criteria to balance the difference between the recorded data
(19) yields (Shepp & Vardi, 1982; Holmes, 1988; Snyder et al., and the reblurred restoration result, given the amount of
1993; van Kempen et al., 1997; van Kempen, 1999) noise present in the image.
kþ1 k k
The method of constrained least squares (CLS) finds
f̂ ¼ f̂ ðHf̂ þ bÞ¹1 HT m ð20Þ a lCLS such that the mean-square-error between the
The EM algorithm ensures a non-negative solution when a recorded data and the blurred restored data equals the
non-negative initial guess fˆ0 is used. Furthermore, the noise power,
X X
likelihood of each iteration of the EM algorithm will strictly 2
g ¹ Hf̂ðlÞ ¼
2
ðI ¹ HAðlÞÞg ¼ e ð23Þ
increase to a global maximum (Snyder & Miller, 1991). The EM
algorithm for finding the maximum likelihood estimator of a with I denoting the unity matrix and e the total noise power.
translated Poisson process (often referred to as EM-MLE) is A is the linear Tikhonov–Miller filter,
identical to the Richardson–Lucy algorithm (Richardson, AðlÞ ¼ ðHT H þ lCT CÞ¹1 HT ð24Þ
1972).
The Richardson–Lucy algorithm is a constrained but We determine lCLS numerically using Brent’s method (Press
unregularized iterative image restoration algorithm. The et al., 1992) to find the zero crossing of the function
X
ICTM and Carrington algorithms, however, incorporate 2
ðI ¹ HAðlÞÞg ¹ e ð25Þ
Tikhonov regularization to suppress undesired solutions.
Conchello has derived an algorithm that incorporates
Tikhonov regularization into the Richardson–Lucy algo-
3.3. The method of generalized cross-validation
rithm (Conchello & McNally, 1996). Incorporation of
Tikhonov regularization yields The method of generalized cross-validation (GCV) is derived
q from the leave-one-out principle (Galatsanos & Katsaggelos,
kþ1
kþ1 ¹1 þ 1 þ 2lf̂ 1992). For every pixel its associated Tikhonov–Miller filter
f̂ regularized ¼ ð21Þ
l is calculated using all but the pixel under consideration. The
with fˆ k þ 1 given by Eq. (20). Using l’Hôpital’s rule it is easy cross-validation function is derived from the mean-square-
to show that (21) becomes (20) when l → 0. We will refer error between the original data and the result after filtering
to this algorithm as the RL–Conchello algorithm. each pixel with its associated Tikhonov–Miller filter. The
generalized cross-validation function (CV),
P
jðI ¹ HAðlÞÞgj2
CVðlÞ ¼ ð26Þ
3. Regularization parameter ½traceðI ¹ HAðlÞÞÿ2
In this section we introduce various methods that have been can be efficiently evaluated in the discrete Fourier domain

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 S TA RT I N G A N D R E G U L A R I Z I N G I M AG E R E S TO R AT IO N 67

(Galatsanos & Katsaggelos, 1992)2, noise power e. Therefore, they require prior knowledge
X 2 4 2 about the noise variance. Often this knowledge is not
l jCðqÞj jGðqÞj
2 2 2
available. It is possible however, to estimate the noise
q ðjHðqÞj þ ljCðqÞj Þ variance using the GCV method (Galatsanos & Katsaggelos,
CVðlÞ ¼ !2 ð27Þ
X ljCðqÞj2 1992). Denoting the value of the regularization parameter
2 2 determined by the GCV method with lGCV, the noise
q jHðqÞj þ ljCðqÞj
variance can be estimated by
The minimum of this function can be found without prior
jðI ¹ HAðlGCV ÞÞgj2
knowledge of the noise variance (Reeves & Mersereau, ĵ2 ¼ ð30Þ
1992). We have determined the minimum of the cross- trace½I ¹ HAðlGCV Þÿ
validation function numerically with Brent’s one-dimensional Although this estimation of the noise variance assumes that
search algorithm (Press et al., 1992). the data are distorted by additive Gaussian noise, the results
presented in this section show that Eq. (30) can also be used
to estimate the average noise variance in a fluorescence
3.4. The method of maximum likelihood
microscopical image distorted by Poisson noise. Using the
An alternative method to determine l without prior experimental conditions as described in section 4 we
knowledge of the noise variance has been named the performed a simulation experiment in which this method
maximum likelihood method (ML) by (Galatsanos & for estimation of the noise variance was tested for confocal
Katsaggelos, 1992). It is based on a stochastic approach, microscopical imaging.
which assumes ulCf and the noise to be Gaussian Figure 2 shows the relative error (the estimated value
distributed. The former assumption can be met with a minus the true value divided by the true value) of the
proper choice of the regularization matrix C (Galatsanos & estimated noise variance in the top-left graph. The top-right
Katsaggelos, 1992). The derived maximum likelihood graph shows the relative error in the regularization value
function, determined with the CLS method using the estimated and
gT ðI ¹ HAðlÞÞg true variance value. We define the true value of the
MLðlÞ ¼ ð28Þ variance as the mean variance over the image, which in
ðdet½I ¹ HAðlÞÿÞ1=M
the case of Poisson noise is equal to the mean intensity of
can be evaluated in the discrete Fourier domain (Galatsanos the acquired image. The bottom graph shows the relative
& Katsaggelos, 1992) difference between the mean-square-error performance of
X ljCðqÞj2 jGðqÞj2 the ICTM algorithm with the regularization parameter
determined with the CLS method using an estimated
jHðqÞj2 þ ljCðqÞj2
MLðlÞ ¼  q 1=Q ð29Þ variance and the true value of the variance. From Fig. 2
ljCðqÞj2 we conclude that using the estimated variance produces
∏
q jHðqÞj2 þ ljCðqÞj2 values of the regularization parameter very close to those
found by the CLS method using the true value for the
with Q the number of Fourier coefficients. We used Brent’s
variance. Furthermore, it is shown that the mean-square-
one-dimensional search algorithm to find the minimum of
error performance of the ICTM algorithm is hardly
(29).
influenced by using the estimated variance.
The values of the regularization parameter as determined
3.5. The Golden search method by SNR, CLS, GCV and ML are plotted in Fig. 3 as a function
We compare these four methods for determining l (SNR, of the signal-to-noise ratio. We define the optimal value of
the regularization parameter as the value that minimizes
CLS, GCV and ML) with the ‘optimal’ value for l. We define
the mean-square-error between the original image f and the
the optimal value of l as the value for which the image
restored image f̂. We have determined the optimal values of
restoration algorithm (either the ICTM algorithm or
the regularization parameter with a Golden search algo-
Carrington’s algorithm) produces the smallest mean-
rithm (Press et al., 1992) for both the ICTM algorithm and
square-error. We have used the Golden search algorithm
the Carrington algorithm. These values are also plotted in
to find the optimal value numerically (Press et al., 1992).
Fig. 3.

3.6. Estimation of the noise variance

4. Comparison of the regularization parameter methods
Both the CLS method and the SNR method use the total
The simulation experiments described in this section are
2
See Golub et al. (1979) for a step-by-step derivation of the generalized cross performed on images with a size of 128 × 128 × 64 pixels. We
validation function. used a two times oversampled confocal point spread function,

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

68 G . M . P. VA N KEMPEN A ND L. J. VA N VLIET

Fig. 2. The top left graph shows the relative

error of the estimated variance using GCV.
The top right graph shows the relative
error of the regularization parameter deter-
mined with CLS using the variance esti-
mated by GCV. The bottom graph shows
the relative difference in mean-square-
error of ICTM with CLS regularization due
to estimation of the variance.

an NA of 1.3, an excitation wavelength of 480 nm, an

CLS emission wavelength of 530 nm3, a refractive index of 1.515
GCV and a backprojected pinhole size of 300 nm. Spheres with a
SNR diameter of 1000 nm were used as objects. The sampling
1
ML
distance was 25.0 nm lateral and 85.0 nm axial. We varied
the signal-to-noise ratio (as defined by the ratio of the object
Golden (ICTM)
regularization parameter

0.1
energy and the noise energy) from 1.0 to 256.0 (0.0 dB to
Golden (Carrington)
24.1 dB), which corresponds to a conversion factor ranging
from 0.14 to 35.8, with an object intensity of 200.0 ADU
0.01 and a background of 20.0 ADU.
We have based our implementation of the various
methods for determining the regularization parameter on
0.001
sources provided by Peter J. Verveer. We reimplemented
these algorithms in the scientific image processing library
0.0001 DIPlib (www.ph.tn.tudelft.nl/DIPlib, Pattern Recognition
Group, Delft University of Technology, The Netherlands) to
incorporate the methods to estimate the signal-to-noise
1E-05
1 10 100 ratio as presented in section 3.6.
signal to noise ratio Figure 4 shows the mean-square-error performances of
the ICTM and Carrington algorithms for the various
Fig. 3. The value of the regularization parameter, using SNR, CLS,
GCV and ML, as a function of the signal-to-noise ratio. The optimal
3
values (Golden) of l for the ICTM and Carrington’s algorithm are These excitation and emission wavelength correspond to the excitation and
also plotted. emission spectra of popular fluophores like FITC and Nile-Red.

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 S TA RT I N G A N D R E G U L A R I Z I N G I M AG E R E S TO R AT IO N 69

Fig. 4. The value of the mean-square-error of the restoration result of the Carrington (left) and ICTM (right) algorithm. The regularization
parameter was determined with the CLS, GCV, SNR and ML method and a Golden search. The performance of an unregularized ICTM is also
shown.

methods used to estimate the regularization parameter. We parameter, which could not be handled properly by the
have also included the performance of the ICTM with no Carrington algorithm. We attribute the low performance of
regularization (l set to zero). Owing to numerical instability the Carrington algorithm in this case to numerical
(see also the low performance of the Carrington algorithm instability of the algorithm (we believe that this is related
for low values of l as estimated with ML), we were not able to the ‘extra’ transform (Eq. (12)) that the Carrington
to include these results for the Carrington algorithm. All the algorithm incorporates).
results shown in Fig. 4 are averages over eight experiments. The performances of the GCV and the Golden search
All data points have a coefficient-of-variation of less than method are equal within the accuracy of the experiment.
5%. Clearly the GCV method is our method of choice for
Figure 4 clearly shows that high values of the regulariza- estimating the regularization parameter.
tion parameter as determined by the SNR method result in a Figure 5 shows centre x-y and x-z slices of the restoration
low performance caused by an oversmoothed restoration results obtained with the ICTM and Carrington algorithms
result (see Fig. 5). using the different methods to determine the regularization
The CLS method is also known to overestimate the value parameter. At first, the result obtained with the SNR
of the regularization parameter (Galatsanos & Katsaggelos, method might seem the most appealing since it lacks the
1992). Therefore, the CLS method will produce some- structures or texture present in the other results that are
what smooth (restored) images resulting in a suboptimal not present in the original object. These structures are
performance. caused by the noise realization in the acquired image. The
The Tikhonov functional (see Eq. (7)) includes a SNR method imposed the strongest regularization at the
regularization matrix R, which is supported by the ICTM SNR chosen in this figure (see Fig. 4), which smooths not
algorithm. The Carrington algorithm does not support this only the noise-induced texture but also the restoration
regularization matrix. We have therefore chosen to make R result.
equal to the unity matrix I and not to use a specific
regularization matrix R that would make Rf Gaussian
5. The regularization parameter of the RL–Conchello
distributed, as required by the ML algorithm. This explains
algorithm
the weak performance of the ML method in combination
with the ICTM algorithm. In our experiments the ML In the previous sections we have shown that the GCV
method produces very small values for the regularization method is a good method for determining the regularization

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

70 G . M . P. VA N KEMPEN A ND L. J. VA N VLIET

Fig. 5. Restoration results of the ICTM and Carrington algorithms with the regularization parameter determined by the SNR, CLS, GCV, ML,
and Golden search method. The pictures show the centre x-y and x-z slices of the centre (80 × 80 pixels x-y, 80 × 40 pixels x-z) of an image of
a sphere with a 0.50-mm radius and a signal-to-noise ration of 8.0.

parameter for the ICTM algorithm and the Carrington A second order Taylor series expansion of ??? around g
algorithm. The method is computationally feasible as we use yields
the Tikhonov–Miller algorithm to compute an estimate for f. X X X
The RL–Conchello algorithm is based on a regularized ð2mÞ¹1 ðHf̂ ¹ gÞ2 þ lkf̂k2 ¼ ðHf̂ þ gÞ2 þ 2l mf̂ 2
maximum likelihood function for images distorted by ð34Þ
Poisson noise. No linear solution exists for this functional.
To be able to find a linear solution of Eq. (34) in the form of
We have therefore approximated the functional such that it
a Tikhonov–Miller filter, we need to approximate m by a
can be solved linearly. This linear solution can then be used
constant. Using the mean of m Eq. (34) becomes
to estimate the regularization parameter with a feasible X
computational complexity. ðHf̂ ¹ gÞ2 þ 2lm̄kf̂k2 ð35Þ
The RL–Conchello maximizes the regularized loglikeli-
hood functional of a translated Poisson process Using one of the methods to determine the regularization
X T
parameter for the Tikhonov–Miller filter, we can determine
Lðf̂Þ ¼ ¹ Hf̂ þ m lnðHf̂ þ bÞ ¹ lkf̂k2 ð31Þ the regularization parameter for the RL–Conchello algorithm
Maximizing Eq. (31) is equal to minimizing lRL¹Conchello ¼ lTM =2m̄ ð36Þ
X T
cþ Hf̂ ¹ m lnðHf̂ þ bÞ þ lkf̂k2 ð32Þ Thompson (1989) has used similar approximations for
determining the regularization parameter for a mean-
where c is a scalar constant. If we choose c to be the log square-error functional regularized by an entropy measure.
likelihood of the recorded data, Eq. (32) becomes Figure 6 shows the values of the regularization value for
X X T the RL–Conchello algorithm determined by the CLS, GCV,
g ¹ mT ln m þ Hf̂ ¹ m lnðHf̂ þ bÞ þ lkf̂k2 ð33Þ SNR and ML algorithms. Again we used the Golden search

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 S TA RT I N G A N D R E G U L A R I Z I N G I M AG E R E S TO R AT IO N 71

algorithm to find the optimal value of the regularization (this imposes a smoothing comparable to the smoothing of
parameter, which is also included. the point spread function used in this experiment).
The mean-square-error performance of the RL–Conchello
algorithm is shown in Fig. 7 with the regularization
X a homogeneous noise image
parameter determined by the five different methods as a
function of the signal-to-noise ratio. We have also included The mean of g (or m) is a constant image in the spatial
the performance of the Richardson–Lucy algorithm, the domain and an impulse in the Fourier domain. In that case
unregularized version of the RL–Conchello algorithm. the restoration algorithms have to restore the complete
It is clear from Fig. 6 that the optimal value of the Fourier domain starting from an impulse. It is interesting to
regularization value is determined considerably less accu- see if it is beneficial to start with a first estimate that fills the
rately by any of the four proposed methods than in case of Fourier domain. We have generated such an image by
the ICTM and Carrington algorithms. One reason for this generating a constant image distorted by Poisson noise. The
could be the approximation we had to make in the mean of this image has been set to the mean of g.
derivation of Eq. (35), where we approximated m by m̄.
However, the GCV method still produces results that are
X the result of Tikhonov-Miller restoration
considerably better in the mean-square-error sense than the
results of the unregularized Richardson–Lucy algorithm. The iterative algorithms based on the Tikhonov
functional should converge more quickly to their solutions
if a first estimate close to the final solution is provided. We
6. First estimates
used the restoration result produced by the linear
The investigated iterative restoration algorithms need a first Tikhonov–Miller filter to test this hypothesis. For the RL–
estimate to start their iterations. In this section we compare Conchello and Richardson–Lucy algorithms we have
the performance of the ICTM, Carrington and Richardson– clipped this result at zero to guarantee the positivity of the
Lucy algorithms using different first estimates. We have first estimate.
tested the following first estimates:
X the original object
X the recorded image
Ideally, the restoration algorithms should produce the
An obvious choice for the first estimate is to use the image original object. Using the original object as a first estimate
m acquired by the microscope. For the ICTM and enables us to test whether this is the solution found by these
Carrington algorithms we have subtracted the (constant) algorithms under realistic circumstances.
background b from m to make the summed intensity of
(m ¹ b) equal to that of the original object. We have defined
m ¹ b as g (see Eq. (2)). For the RL–Conchello and
Richardson–Lucy algorithms, we used m to guarantee the CLS

positivity of the first estimate. 0.01 GCV

SNR
regularization parameter

X the mean of the recorded image ML

0.001
Golden
Kaufman (1987) has argued that starting from a uniform
first guess is a good approach if no other reasonable guess as
0.0001
a first estimate can be obtained. We have used the mean of g
for the ICTM and Carrington algorithms and the mean of m
for the RL–Conchello and Richardson–Lucy algorithms as a
1E-05
uniform first estimate.

X a smoothed version of the recorded image 1E-06

The recorded image contains the smoothed object as well as 1 10 100

noise. To prevent the restoration algorithm adapting to signal to noise ratio

noise realizations in the recorded image one could reduce Fig. 6. The value of the regularization parameter for the RL–
the noise in g (m for the RL–Conchello and Richardson– Conchello algorithm determined with SNR, CLS, GCV and ML, as
Lucy algorithms) by means of local smoothing. We have a function of the signal-to-noise ratio. The optimal values (Golden)
smoothed g with a Gaussian filter with a sigma of 2.0 pixels are also plotted.

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 72 G . M . P. VA N KEMPEN A ND L. J. VA N VLIET

CLS IC TM
GCV 80
recorded im age
100 SNR m ean image
sm oothed im age
ML Poisson noise
Tikhonov-M iller
mean square error

Golden 60 original object

m ean square error

Unregularized

10
40

20
1

1 10 100
signal to noise ratio 0
1 1 0 100

Fig. 7. The value of the mean-square-error of the restoration result iteration

of the RL–Conchello algorithm. The regularization parameter is
determined with the CLS, GCV, SNR and ML method and a Golden ICTM
search. The performance of the unregularized Richardson–Lucy 0.6
recorded image
algorithm is also shown. mean image
smoothed image
X a zero image for the Carrington algorithm only 0.5 Poisson noise
Tikhonov-Miller
The Carrington algorithm finds the estimate of f by original object

optimizing a functional as function of a transformed 0.4

I-divergence

parameter. Therefore, none of the first estimates proposed

above is a really good estimate for the Carrington algorithm. 0.3

Carrington has proposed the use of an image set to zero as

the first estimate (Carrington, 1990). 0.2
We have measured the mean-square-error and the
I-divergence of the ICTM algorithm, the Carrington algo-
0.1
rithm, the RL–Conchello algorithm and the Richardson–
Lucy algorithm as a function of the number of iterations for
the various first estimates. The measured mean-square- 0.0
1 10 100
error and I-divergence values of the first 200 iterations are iteration
shown in Fig. 8 for the ICTM algorithm, in Fig. 9 for the
Carrington algorithm, in Fig. 10 for the RL–Conchello Fig. 8. The mean-square error and I-divergence for different first
algorithm, and in Fig. 11 for the Richardson–Lucy estimates as a function of the number of iterations performed by
algorithm. The data show the averages over eight the ICTM algorithm.
repetitions of the experiment using different noise realiza-
tions. Note that we have used different scales in these the first estimates is a really good first estimate for the
graphs to account for the differences in the range of the Carrington algorithm. The convergence is slow and rough
data. when compared to the other results.
From these figures we conclude that the ICTM and RL– The three regularized algorithms (ICTM, Carrington and
Conchello algorithms using the various first estimates RL–Conchello) converge to a result that is very similar in
converge to the same end result in both mean-square-error both the mean-square-error and the I-divergence sense. The
and I-divergence sense. Using the relatively noisy first performance of the (unregularized) Richardson–Lucy algo-
estimates (the recorded image and homogeneous noise rithm is lower than that of the other three and does not
image) the Carrington and Richardson–Lucy algorithms converge to a minimum in both mean-square-error and I-
produce a result with a somewhat poorer performance. Also divergence sense. The performance even decreases somewhat
the zero image first estimate for the Carrington algorithm for a large number of iterations. This can be explained by the
produces a slightly poorer performance. lack of regularization in the Richardson–Lucy algorithm,
The ICTM algorithm shows the fastest convergence of the which allow the restoration result to adapt to the noise
four algorithms tested. The results also show that none of present in the acquired image (Conchello et al., 1994).

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 S TA RT I N G A N D R E G U L A R I Z I N G I M AG E R E S TO R AT IO N 73

Carrington RL-Conchello
recorded image
mean image 300
smoothed image recorded image
Poisson noise mean image
Tikhonov-M iller smoothed image
1000 original object 250 Poisson noise
zero image Tikhonov-Miller
original object
mean square error

10 .6

mean square error

200
10 .4

10 .2

10 .0

150
100 9.8

9.6

9.4
10 0
100
iteration

50
10

1 1 0 100
0
iteration 1 1 0 100
iteration
Carrington
3.5
recorded image RL-Conchello
mean image
3.5
3.0 smoothed image
Poisson noise
Tikhonov-M iller recorded image
2.5 original object 3.0 mean image
smoothed image
Poisson noise
2.5 Tikhonov-Miller
I-divergence

2.0
original object
I-divergence

1.5 2.0

1.0 1.5

0.5 1.0

0.0 0.5

1 1 0 100
0.0
iteration

1 10 100
Fig. 9. The mean-square-error and I-divergence for different first
iteration
estimates as a function of the number of iterations performed by
the Carrington algorithm. Fig. 10. The mean-square-error and I-divergence for different first
estimates as a function of the number of iterations performed by
Only the ICTM and Carrington algorithms benefit from the RL–Conchello algorithm.
using the smoothed image as a first estimate. The ICTM
algorithm converges quickly for this first estimate, whereas priori model of the original (Andrews & Hunt, 1977;
the Carrington algorithm produces a slightly better restora- Verveer & Jovin, 1997). Because the intensities of our
tion result in mean-square-error sense. The convergence of original object are not Gaussian distributed, the algorithms
this first estimate is worse than that of the recorded image for will find a solution, which is a balance between the distance
both the RL–Conchello and the Richardson–Lucy algorithm. to the original object and the Gaussian model imposed by
Using the restoration result of the Tikhonov–Miller filter as a the regularization.
first estimate gives by far the fastest convergence for all Although the result was a surprise, we have not
algorithms except for the Carrington algorithm. investigated why the ICTM algorithm produces a
Even when the original object is used as a first estimate, maximum mean-square-error at about 10 iterations when
the various algorithms converge to the same solution in the the original object is used as a first estimate. This
mean-square-error sense as obtained with the other first phenomenon could be related to the type of object or
estimates. This can be explained by the regularization. The the type of microscope (confocal imaging) used in the
Tikhonov regularization can be regarded as a Gaussian a simulations.

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

74 G . M . P. VA N KEMPEN A ND L. J. VA N VLIET

Richardson-Lucy generalized cross validation (GCV) provides an efficient way

recorded im age
300
m ean image of producing a value of the regularization parameter which
sm oothed im age
Poisson noise
is close to its optimal value. We defined the optimal value to
Tikhonov-M iller be that value that produced the minimum mean-square-
250 original object
error between the restoration result and the original object.
50 We determined this optimal value using the Golden search
m ean square error

200
40
algorithm (Press et al., 1992), a numerical minimization
algorithm. Over a large range of the signal-to-noise ratio,

mean square error

30

150 the GCV method produced practically the same value for the
20
regularization parameter as was obtained with this search
100 10 algorithm. (This ‘optimal’ procedure can only be performed
0
on synthetic data as it requires the non-blurred data as
40 60 80 100 120

iteration
140 160 180 200
well.) This indicates that the assumption of additive
50
Gaussian noise and the use of the linear Tikhonov
restoration filter by the GCV method does not influence its
0
performance even when used on images distorted with
1 1 0 100
iteration
Poisson noise at a low signal-to-noise ratio.
We have also derived a method for estimating the
Richardson-Lucy regularization parameter for the RL–Conchello algorithm,
3.5 a Tikhonov regularized version of the Richardson–Lucy
recorded image algorithm. For this algorithm, the GCV method is again the
mean image
3.0
smoothed image best method, among the tested methods, for estimating the
Poisson noise regularization parameter. However, the values obtained
Tikhonov-Miller
2.5
original object with the GCV method are not as close to the ‘optimal’ value
(again obtained with a Golden search), as it was for the
I-divergence

2.0 ICTM and Carrington algorithms. This is probably caused by

the approximation of the acquired image by its mean value,
1.5
which was necessary in the derivation of the regularization
parameter estimation for this algorithm.
1.0
We have investigated the influence of various choices of a
first estimate as used by the iterative image restoration
0.5
algorithms investigated here. We found that most algo-
rithms converged to the same solution for the tested choices.
0.0
Only the Richardson–Lucy algorithm failed to converge to a
1 10 100
steady solution but showed a decrease in performance for a
iteration
large number of iterations. This can be explained by the
Fig. 11. The mean-square-error and I-divergence for different first unregularized nature of this algorithm, making it sensitive
estimates as a function of the number of iterations performed by to the noise realization present in the acquired image.
the Richardson–Lucy algorithm.

7. Conclusions Acknowledgements

The performance of non-linear iterative image restoration This work was partially supported by the Royal Netherlands
algorithms is dependent on various parameters. In this Academy of Arts and Sciences (KNAW) and by the Rolling
paper we have investigated the impact of the regularization Grants program of the Foundation for Fundamental
parameter and the choice of first estimate. Research in Matter (FOM).
We have compared various methods for determining the
regularization parameter, which has been developed by References
Galatsanos (Galatsanos & Katsaggelos, 1992) for linear
Andrews, H.C. & Hunt, B.R. (1977) Digital Image Restoration.
Tikhonov restoration of images distorted with additive Prentice Hall, Englewood Cliffs, NJ.
Gaussian noise. We have tested the ability of these Carrington, W.A. (1990) Image Restoration in 3D microscopy with
algorithms to determine the regularization parameter of limited data. Bioimaging and Two-Dimensional Spectroscopy. Proc.
non-linear restoration algorithms for the restoration of SPIE Vol. 1205 (ed. by L. C. Smith), pp. 72–83. SPIE, San Jose.
images distorted with Poisson noise. Our results show that Carrington, W.A., Lynch, R.M., Moore, E.M., Isenberg, G., Fogarty,
for the ICTM and Carrington algorithms the method of K.E. & Fay, F.S. (1995) Superresolution three-dimensional

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75

 S TA RT I N G A N D R E G U L A R I Z I N G I M AG E R E S TO R AT IO N 75

images of fluorescence in cells with minimal light exposure. Lagendijk, R.L. & Biemond, J. (1991) Iterative Identification and
Science, 268, 1483–1487. Restoration of Images. Kluwer Academic Publishers, Dordrecht.
Conchello, J.A., Kim, J.J. & Hansen, E.W. (1994) Enhanced 3- Miller, K. (1970) Least squares methods for ill-posed problems with
dimensional reconstruction from confocal scanning microscope a prescribed bound. SIAM J. Math Anal. 1, 52–74.
images. 2. Depth discrimination versus signal-to-noise ratio in Press, W.H., Teukolsky, S.A., Vetterling, W.T. & Flannery, B.P.
partially confocal images. Appl. Optics, 33, 3740–3750. (1992) Numerical Recipes in C. 2nd edn. Cambridge University
Conchello, J.-A. & McNally, J.G. (1996) Fast regularization Press, Cambridge.
technique for expectation maximization algorithm for optical Reeves, S.J. & Mersereau, R.M. (1992) Blur identification by the
sectioning microscopy. Three-Dimensional Microscopy: Image method of generalized cross-validation. IEEE Trans. Image
Acquisition and Processing III Vol. 2655 (ed. by C. J. Cogswell, Processing, 1, 301–311.
G. S. Kino and T. Wilson), pp. 199–208. SPIE, San Jose. Richardson, W.H. (1972) Bayesian-based iterative method of image
Dempster, A.P., Laird, N.M. & Rubin, D.B. (1977) Maximum restoration. J. Opt. Soc. Am. 62, 55–59.
likelihood from incomplete data via the EM algorithm. J. Roy. Shepp, L.A. & Vardi, Y. (1982) Maximum likelihood reconstruction
Statist. Soc. B, 39, 1–37. for emission tomography. IEEE Trans. Med. Imaging, MI-1, 113–
Galatsanos, N.P. & Katsaggelos, A.K. (1992) Methods for choosing 121.
the regularization parameter and estimating the noise variance Snyder, D.L., Hammoud, A.M. & White, R.L. (1993) Image
in image restoration and their relation. IEEE Trans. Image recovery from data acquired with a charge-coupled-device
Processing, 1, 322–336. camera. J. Opt. Soc. Am. A, 10, 1014–1023.
Golub, G.H., Heath, M. & Wahba, G. (1979) Generalized cross Snyder, D.L. & Miller, M.I. (1991) Random Point Processes in Time
validation as a method for choosing a good Rigde parameter. and Space. Springer Verlag, Berlin.
Technometrics, 21, 215–223. Thompson, A.M. (1989) On the use of quadratic regularisation
Holmes, T.J. (1988) Maximum-likelihood image restoration adapted within maximum entropy image restoration. Maximum Entropy
for noncoherent optical imaging. J. Opt Soc. Am. A, 5, 666–673. and Bayesian Methods (ed. by J. Skilling), pp. 497–504. Kluwer
Kaufman, L. (1987) Implementing and accelerating the EM Academic Publishers, Dordrecht.
algorithm for positron emission tomography. IEEE Trans. Med. Tikhonov, A.N. & Arsenin, V.Y. (1977) Solutions of Ill-Posed
Imaging, MI-6, 37–51. Problems. Wiley, New York.
van Kempen, G.M.P. (1999) Image Restoration in Fluorescence Vardi, Y., Shepp, L.A. & Kaufman, L. (1985) A statistical model for
Microscopy. PhD Thesis, Delft University of Technology, Delft, The positron emission tomography. J. Am. Statist Assoc. 80, 8–35.
Netherlands. Verveer, P.J., Gemkow, M.J. & Jovin, T.M. (1999) A comparison of
van Kempen, G.M.P., van Vliet, L.J. & Verveer, P.J. (1997) image restoration approaches applied to three-dimensional
Application of image restoration methods for confocal fluores- confocal and wide-field fluorescence microscopy. J. Microsc.
cence microscopy. 3-D Microscopy: Image Acquisition and Proces- 193, 50–61.
sing IV Proceedings of the SPIE. Vol. 2984 (ed. by J.-A. Conchello, Verveer, P.J. & Jovin, T.M. (1997) Efficient super-resolution
C. J. Cogswell and T. Wilson), pp. 114–124. SPIE, San Jose. restoration algorithms using MAP estimations with application
van Kempen, G.M.P., van Vliet, L.J., Verveer, P.J., van der Voort, to fluorescence microscopy. J. Opt Soc. Am. A, 14, 1696–1706.
H.T.M. (1997) A quantitative comparison of image van der Voort, H.T.M. & Strasters, K.C. (1995) Restoration of
restoration methods for confocal microscopy. J. Micros. 185, confocal images for quantitative image analysis. J. Microsc. 178,
354–365. 165–181.

q 2000 The Royal Microscopical Society, Journal of Microscopy, 198, 63–75