Invited Paper

Obtaining and Reproduction of Accurate Color Images

Based on Human Perception
Yoichi MIYAKE and Yasuaki YOKOYAMA

Department of Information and Computer Sciences,

Chiba University, Inage-ku, Chiba 263, Japan

ABSTRACT
We have developed high definition color imaging systems for digital artworks. The system is consisted of multiband camera,
personal computer and projection type monitor. The multiband camera is composed by five band filters and single chip CCD
camera with 2048x2048 pixels. On the basis of principal component analysis and Wiener estimation method, the reflectance
spectra of each pixel of the object were estimated using five band images taken by CCD camera. Images of paintings are
reproduced onto both the projection type monitor and CRT under the consideration of color appearance.
In this paper, method to estimate of reflectance spectra of paintings from five band images is described, and the color
reproduction of paintings based on the color appearance models is also discussed.

Keywords: principal component analysis, color reproduction, spectral reflectance, color adaptation

1. INTRODUCTION
In recent years, high quality digital color imaging system is required in the field of artworks, medical imaging and
printings instead of conventional photographic systems with an advent of information superhighway Digital imaging system
is consisted of image capturing, store and display components in different from photographic system. Figure 1 shows the
schematic diagram of developed multiband color imaging system. The quality of digital imaging is not only influenced by the
characteristics of each component but also an image processing technology. As well known, image quality is determined by
tone reproduction, sharpness, noise and color reproduction characteristics. Tone, sharpness and noise of the image which are
evaluated by dynamic range, SIN, a number of pixels are mainly caused by the abilities of CCD area sensor. On the other
hand, color reproduction of the imaging systems is considerably influenced by color management characteristics. In this
paper, we discuss only the color reproduction of the digital imaging systems.

2. ESTIMATION OF REFLECTANCE SPECTRA OF THE OBJECT

2.1 Principal Component Analysis
For designing of high accurate color reproduction systems, it is important to know the spectral characteristics of each
imaging component such as lens, filter, CCD and also to know the spectral reflectance of the object. The characteristics of
color imaging system V1 can be expressed by
= JE(A)L(2)S(AlO(A)f(2k-JA (i = 1,2 5) (1)

where E(A) is spectral radiant distribution of illuminant, O(2) is spectral reflectance of the object, S() is spectral sensitivity

190 SPIE Vol. 3300 . 0277-786X/981$10.00

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 - -

Image A.qutsiiion Ssiem Image database

E( ?.>
I )isplav Managemeni
System
v.
S(X)
Spectra Color Appearance
Lsti m anon Transformation

Art paint gital camera PC Color Space

________________
(A0. 0 Filterf )
Viewing conditions
1 ranstormaiion

- t)cvicc
p of des

cohr appearance J

Display device
Colon metric cob reprod IIC lion Observer
(K0. }0.Z0) fAd. dZd) r Aj. Zj
Figure 1 Schematic diagram ot developed multiband color imaging system.

of CCD. F')L). LCL) is spectral transmittance of imaging filters and lens respectively. Therefore it can he done easily the
computer simulation of color reproduction for digital imaging system. In the computer simulation of color reproduction
based on Eq. I >, however. ()2
is processed as one dimensional signal. For high accurate simulation, it is necessary to know
the two dimensional (entire pixels) spectral reflectance of the object O(x,v, A). where (xv) is coordinate of object. For this
purpose we measured the spectral reflectance of many kinds of color paints and those reflectance spectra were analvied by
the principal component analsis. Eq.) I) can he expressed as Eq.(2) by using vector matrix notationL
% =JELSO
= ibo = t.:. (2)
wheref E. L are vector notation of spectral characteristics of filter. illuniinant. lens and CCI) respectively. F is the
overall spectral characteristics of imaging system. The reflectance spectra of the object 0 can he represented by principal
component analysis as

0=au +m ()
,=1

where (1 is an expansion coefficient and in is mean ector of reflectance spectra of oil paint samples. From the pri nc pal
component analysis of the oil paintins. () can he approximately hr using a linear combination of live cigeflveclors. U ii iI.
Thus the Eq.(2t can he expressed as

I. a
V.
=
a.
U. U] (4)

L 1[U +fl1J

191

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 The expansion coefficients a1a2...cz5 are calculated by
.. . Fu5 '
a1 FitUi FitU2 %' "im

a2 F2:ui jt .
F2tu5
% (5)

a5 F5tu1 F;tu2 F;tu5 vs m

where
m
V2m
= F2t
. m (6)

V5m Ft

Figure 2 shows the five principal components of the reflectance spectra of paintings. Figure 3 shows the cumulative contribu-
tion ratio of the principal components. From this result, we can calculate the reflectance spectra of all pixels of the object
O(x,y, A) from the five band images Vi(x,y) (i=1,2 5) taken by single chip high quality CCD camera (Dainippon Screen,
Fotex F-1O : 2048x2048 pixels 12 bits AID, 8 bits output) with rotating color wheel comprising five color filters.

C
£T__ .
, X*-...-x -
-
0 ....w...
-
OtT
)
0

_"•oo 500 600

wavelength [nm]
700 4X) 500 600
wavelength [nm]
700 i
>

E
o.g;
.

2nd
1st C) ç-
',
: . 4 5
1

I/. number of principal components

7/
:\J Figure 3 Cumulative contribution ratio

500 600700 400 500 600 700

wavelength [tim] wavelength[nm]
3rd 4th

,'
I/\. f\
.1I '\l \ I

.0.5
4(X) 5(X) 600 700
wavelength [nmj
5 th wavelength [nm]
Figure 2 Five major principal components of Figure 4 An example of estimated reflectance spectrum.
reflectance spectra of paintings.

192

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 2.2 Wiener Estimation of Reflectance Spectra
In order to estimate of reflection spectra of the object, we also used Wiener estimation method proposed in a previous
paper2. Spectral reflectance of the object °estcan be estimated by
Oesz=GV (7)
where V denotes vector notation of camera response V . We don't describe detail here on Wiener estimation method. The
Matrix G in Eq.(7) is determined to minimize the ensemble average of the square error between original and estimated
spectral reflectance, where < > denotes ensemble average.

E ((Oreat Oest )t(O — Oest )) (8)

Namely, G can be expressed as,

G = ROVRV' (9)
where R0 and R are correlation matrices defined as
R0 = (OVt), R = (VV) (10)
In our experiment, correlation matrices were calculated by using the reflectance spectra of 147 oil paintings and the color
difference in CIEL*a*b* color space. In Wiener estimation, the accuracy of estimation is largely dependent on the shape of
filter used in multiband camera, namely the spectral transmittance characteristics of color filters.
In next section, we describe on the filter optimization for five band image capturing.

3. DESIGNING OF SEPARATION FILTERS

In Wiener estimation of reflectance spectra using the multiband image, the accuracy of the estimation is dependent on the
spectral transmittance characteristics of the separation filters. In designing of filter, various parameters must be optimized
such as a number of filter, peak wavelength, half-band width, shape of spectral transmittance etc. These are very complex
problems to solve. Therefore, we assumed that a filter number is five based on the principal component analysis, and the
spectral transmittance of filters has Gaussian distribution as

f()=exP[_)] (i=1,2 5) (11)

where .a peak wavelength, a is half-band width of filter. The peak wave-length and half-band width were estimated by
Wiener estimation method and optimized by using the simulated annealing method. In the optimization of filters, the color
difference between actual and estimated spectra in the CIEL*a*b* color space was used as a cost function. The optimized
spectral transmittance of five filters is shown in Figure5. Figure 6 shows a schematic diagram of the optimization process.

1.0

.
—
0.8
0.6

I\Q
c,

0.2
0.0 - -- ..
400 450 500 550 600 650 700
wavelength [nm]

Figure 5 Optimal spectral transmittance of five filters.

193

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 Figure 6 Schematic diagram of filter optimization process.

*1

sample number

Figure 7 Estimation error (color difference).

i
•1

o.o
400
:•
;i.i:i.u_,
500
'600 700
wavelength [nm]
Figure 8 The best combination of gelatine filter set.

The color difference between actual and estimated of 147 oil paintings by using optimal filter was less than 0.62(average
0.22) as shown in Fig.7. However, it is not easy to make such filters in practice. Then, we found five filters to minimize the
color difference from the 24 kinds of practical gelatin filter set (FUJI FILM TAC Optical Filter), namely n=24! / [5!x(24-5)!]
= 42,504 combinations of filter, by computer simulation. Figure 8 shows those obtained spectral transmittance of five filters.
Average color difference AE*ab =1.16 was obtained in the estimation of reflection spectra of 147 oil paintings using five
band images taken by these filters.

194

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 hand #1 hand #2 band #3 hand #4
Figure 9 Muttihand images.

/___=, r

Figure 10 Synthesized image (Illuminant D05).

4. COLOR REPRODUCTION
As we could estimate the reflection spectra of all pixels in the object. it is easy to do computer simulation ha'cd on
colorimetric color reproduction. As shown in Eq.( 1). we can estimate the color reproduction of Images taken by different
illununants or spectral sensitivity of CCD sensors or spectral transmittance of lenses. This simulation was significant to esti
mate optimum illuminant for electronic endoscopes and to predict skin color images under various illuminants. E{o cer. a
well known, the perceived color is different from the colorimetric color. Therefore it is important to introduce the color appcat
ance for color management. The mechanism of perceived color and human perception is very complex problem. In this study.
e only considered chromatic adaptation for various ' iewing illuminants. Many chromatic adaptation models ha c hceti
proposed by (in Kries. Hunt. Nayatani. Fairchild. In our pre ious papers. we compared those models to estimate the skin
color reproduction under various illuminants h psvchoph\ steal experiment based on memory matching method4 . From this
experiment. e found that the incomplete chromatic adaptation model proposed hr Fairchild v as apparentlr etlccii c (or
prediction of skin color patch. However. Fairchild model as not always e0etis c to predict the color reproduction of facial
pattern as same as other models. Therefore ss e improved Fairchild color adaptation model to predict the color reprodii.t oil of
facial pattern precisely and applied it to the reproduction of paintings. In Fairchild model. L' NI'. S' the cone fuiidatiicniai
values of after adaptation can he expressed by

L =p1L/L5
t1, = pt1,I .i1
S ç.S/5\

195

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 where L,M,S are cone lundamental values before adaptations, are cone fundamental responses for the white
point of the illuminant. are the coefficients of incomplete adaptation calculated as

PL = (1 + + l >1(1 + } +
ps., =(l+Y +niE)/(l+}5:+1/niE) (13)
ç =(l + Y )( + } + 1/SE)
where Y\ is the luminance of the illuminant. e is an exponent to define the degree of the adaptation. and I,. m,s1 are
the fundamental chromaticity coordinates of adaptive stimulus given by

'F =KJLv/(Lv+11v+S\)
K
= L /( L + Al + S (14)
SF=KcL\/(L +Mv+S\)
In the model, color luminance coefficients KIVKSFKc are fixed as 3.0. We considered to change the coefficients and 64 images
with difterent coefficient Ironi 2.9 to 3.2 were prepared. Then the optimum coefficients to predict skin color of facial pn
were selected h rnemor\ matching method under three kinds of ilIuminants Illuminant A(2S37K). Day lightôO47K). and
Cool whitc(3957K). As the result, we concluded that the optimal facial pattern image are reproduced when the coefficients
KL.KrKc equal to 3.1, 2.9.3.1 respectively. Modified Fairchild model with these coefficients, was compared with von Kries.
Fairchild, RLAB and LLAB color adaptation models for three facial patterns under three illuminants. As a result, proposed
model was most significant to predict the color appearance of facial pattern. We introduced the same coefficients to predict

LMS: cone foundamental

tnstimulus value

without adaptation
(colonrnetnc color reproduction)

Display S stem
Projection Type Display
ii:
it h col oi adaptation trail I ormat ion

Figure 11 Color apeaiance transfonnation procedure. Figure 12 Reproduced pictures liluminant A

196

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms

 the color of paintings under various illuminants. Figure 1 1 shows the color reproduction process of our high accurate color
imaging systems. As shown in figure, after the five band images were taken, reflection spectra of paintings are estimated.
Then tristimulus values XYZ before adaptation are calculated, and cone response LMS is obtained by linear transformation.
By using the color appearance model in Eq.(12), the cone response L' M' S' after adaptation can be calculated, then the
tristimulus values X' Y' Z' after adaptation is obtained, and the values are transformed to RGB signals to display the image
onto the CRT or projection type CRT.
5. CONCLUSION

We showed that the spectral reflectance of paintings can be estimated by five principal components. In order to estimate of
reflection spectra of all pixels in the object, we have developed five band CCD camera with 2048x2048 pixels. Spectral
transmittance of five filters used in multiband camera was determined by Wiener estimation and simulated annealing meth-
ods. On the basis of the estimated reflection spectra of the object, a new color management system was introduced which is
considered both colorimetric color reproduction and color appearance. It became possible to record and display of the image
of paintings with high quality by using this system.

6. REFERENCES

1. T.Shiobara, S.Zhou, H.Haneishi, N.Tsumura and Y.Miyake, " Improved Color Reproduction of Electronic
Endoscopes", J.Imaging Sci. Technol., 40(6)494-501(1996)
2. H. Haneishi , T. Hasegawa, N. Tsumura and Y. Miyake, "Design of Color Filters for Recording Artworks",
IS&T 50th Annual Conference, pp. 369-372 (1997).
3. M.D. Fairchild and R.S. Berns, "Image color-appearance specification through extension of CIELAB",
Color Res. Appl., 18, pp. 178-190 (1993).
4. F. H. Imai, N. Tsumura, H. Haneishi and Y. Miyake, "Principal Component Analysis of Skin Color and Its
Appliction to Colorimetric Color Reproduction on CRT Display and Hardcopy", J. Imaging Sci. Technol.,
40(5)422-430 (1996)
5. F. H. Imai, N. Tsumura, H. Haneishi and Y. Miyake, "Prediction of Color Reproduction for Skin Color Under
Different Illuminants Based on Color Appearance Models", J. Imaging Sci.Technol., 41(2) 166-173 (1997)

Further author information -

Prof.Yoichi MIYAKE Yasuaki YOKOYAMA
Dept. of Information and Computer Sciences, Chiba University Graduate School of Science and Technology, Chiba University
1-33 Yayoi-cho, Inage-ku, Chiba 263, Japan 1-33 Yayoi-cho, lnage-ku, Chiba 263, Japan
Tel.:+81-43-290-326l Tel. +8 1-43-290-326
Fax.:+81-43-290-3262 Fax.:+8 1-43-290-3262
E-mail:[email protected] E-mail:[email protected]

197

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/27/2015 Terms of Use: http://spiedl.org/terms