THEFFT: AN ALGORITHM

THE WHOLE FAMILY

CANUSE
The fast Fourier transform is one of the fundamental algorithm families in digital
information processing. The author discusses its past, present, and future, along with its
important role in our current digital revolution.

Ap+peihycooley and Xikey described a recipe deed, the FFT is pcrhaps the most ubiquitous
for computing Fourier coefficients of a time se- algorithm used today to analyze and manipulate
hat used many fewer uiachine operations digital or discrcte data.
an.did the straightfoiward procedure ... What My own research experience with various fla-
lies over the horizon in digital sipial prmcessing is vors of the FFT is evidence of its widc rangc of
anyone's guess, hut1 think it will surprise us all. applicability: electroacoustic music and audio-
.s. Audio Electronics, signal processing, inedical imaging, image pro-
cessing, pattern recognition, computational che-
, ,
mistry, error-correcting codes, spcctral methods
for partial differential equations, and last hut not

T
hese days, it is almost beyond helicf least, mathematics. Of course, I could list inany
that there was a time before digital more applications, notably in radar and cominu-
technology. It seems almnst every- nications, but space and time restrict. E. Oran
one realizes that the data whizzing Brigham's hook is a n excellent placc to start, es-
over the Internet, bustling through our modems, pecially pages two and thrcc, which contain a
or crashing into onr cell phones is ultimately jnst (nonexhanstive) list of 77 applications!'
a sequence of 0's and 1$-a digital scquencc-
that magically makes the world the convenient,
high-speed place it is today Much nf this magic Hlstory
is due to a funily of algnrithins that collectively We can trace the FFT's first appearance, like
go by the name thcfnst Fuurier tmn.for" In- so much of mathematics, hack to Gauss! His in-
terests were in certain astrnnomical calculations
(a recnrrciit area of FFT application) that dealt
1521-96151001~10.0002000IEEE
with the intcrpolation of asteroidal orbits from
a finite set of equally spaced observations. Un-
doubtedly, the prospect of a huge, laborious hand
calculation provided good motivation to dcvelop

60 IN SCIENCE & ENGINEERING

COMPUTING
 Discrete Fourier transforms
The fast Fourier transform efficientlycomputes the discrete Equation 5 that we can rewrite Equation 1 as
Fourier transform. Recall that the DFT of a complex input vec- N --1
tor of length N, X = ( X ( O ) ...,X(N- 1)). denoted f , is an- X^(c,d)= $ W ~ ( c N z + d ) ~ X ( o , b W $ (6)
other vector of length N given by the collection of sums b=O 07.0

The computation is now performed in two steps. First,

(1) compute for each b the inner sums (for ail d)

where WN = e x p ( 2 n f i i N ) . Equivalently, we can view

this as the matrix-vector product FN X< where which is now interpreted as a subsampled DFTof length
N2.Even if computed directly, a t most N1Nz2 arithmetic op-
FN = ((Vik))
erations are required to compute all of the X(b,d). Finally,
is the so-called Fouriermotrix. The DFT is an invertible trans- we compute NlN2transforms of length NI:
form with inverse given by

which requires at most an additional NINIZoperations.

Thus, if computed directly, the DFT would require N’ oper- Thus, instead of (Nl N2)‘ operations, this two-step approach
ations. Instead, the FFT is an algorithm for computing the uses at most (NI N2)(Nl + N2)operations. If we had more
DFT in O(N log N) operations. Note that we can view the factors in Equation 6, then this approach would work even
inverse as the DFT of the function better, giving Cooley and Tukey‘s result. The main idea is
that we have converted a 1 D algorithm, in terms of index-
ing, into a 2D algorithm. Furthermore, this algorithm has
the advantage of an in-place implementation, and when
so that we can also use the FFT to invert the DFT. accomplished this way, concludes with data reorganized
One of the DFT‘s most useful properties is that it converts according to the well-known bit-reversal shuffle.
circular or cyclic convolution into pointwise multiplication, This “decimation in time” approach is one of a variety of
for example, FFT techniques. Also notable is the dual approach of ”deci-
mation in frequency” developed simultaneously by Cordon
X 3 ( k ) = f(k)Y^(k) Sande, whose paper with W. Morven Gentleman also con-
(3)
where tains an interesting discussion on memory consideration as
it relates to implementational issues.’ Charles Van Loan‘s
(4) book discusses some of the other variations and contains an
extensive bibliography.’ Many of these algorithms rely on
Consequently, the FFT gives an O(N log N) (instead of an N*) the ability to faclor N. When N is prime, we can use a differ-
algorithm for computing convolutions: First compute the ent idea in which the DFT is effectively reduced to a cyclic
DFTs of both X and U, then compute the inverse DFT of the convolution instead.’
sequence obtained by multiplying pointwise x^ and Y^ .
In retrospect, the idea underlying the Cooley-Tukey FFT is References
quite simple. If N= NiN2, then we can turn the 1 D equation 1. W.M. Gentlemanand C. Zande, ”Fast Fourier Transformr-For Fun and
(Equation 1) into a 2D equation with the change of variables Profit,” Pm. ioliloint Computer Cont ANPS, Vol. 29, Spamn, Warhingtan,
D.C., 1966, pp, 563-578.
j=j(a,S) = aN1t b, 0 20 < Nz,02 b <NI 2. C. Van Loan, ComputotionalFramework tarthefart buier rmnriorm, SIAM,
k = k(c,d) = CN’t d, 0 2 c < NI,0 5 d < Nz (5) Philadelphia, 1992.
3. C.M. Rader, “Discrete FaurierTranrformrWhen the Number of Data Point5 is
Using the fact W;“’ = W;W/ , it follows quicklyfrom Prime,” Pmc I€€€, IEEE Pres, Piscalpway, NJ.. Val. 56,1968,pp. 1107-1 108.

a fast algorithm. Fcwer calculations also imply Fourier t r m s ~ o r m sof length NI, which are c o n -
less opporhillity for error and tbcrcfore lcad to bioed as NI DlTs o f l c n g t h N2.(See the “Dis-
iiuinerical stability. Gauss obscrved that hc could crete Fouricr transforms” sidcbar for (letailcd
break a Fourier series of handwidth N = NlN2 infcirmation.) Gauss’s algoritlim was iievcr pub-
i n t o a c o i q n i t “ t i o n of N2subsampled discrete lished outside of his collccted works.
 T h e statistician FrankYates pul&licd a less gen- win’s fervent proselytizing, did a lot to publicize the
eral hut still important version of the FET in 1932, existence of this (apparently) new fast algorithm.“
which wc can usc to efficiently ctitnpnte the T h e timing of the announceiuent was such that
Harlamard and Walsh uansf~irtns?Yates’s “inter- usage spread quickly T h e ronghly simultaneous de-
action algorithm” is a East technique designed to vclopmcnt of analog-to-digital converters capable
compute thc analysis of variance for a 2”-factorial of producing digitized samples of a time-varping
design and is dcscribed in almost any text on statis- voltage at rates of 300,000 samples per sccond had
tical design and analysis of expcriments. already initiated something of a digital revolution.
Another important predecessor is tlie work o f This devclopinent also provided scientists with
G.C. Danielson and Cornelius Lanczos, per- heretofore uniniagined quantities of digital data to
Conned in the service of x-ray ciystallography, an- analyze and nnnipulate (just as is thc case today).
other area for applying FYI’ techno lo^.' -17beir T h c “standard” applications of FFT as an analysis
“doubling trick” showed how to reduce a DFI‘on tool for waveforms or for solving PDEs generatcd a
2 N points to two DIiTs on N points using only N treniendons interest in the algorithm a priori. nut
extra operations. Today, ith amnsing to notc their moreover, thc ability to do this analysis quickly let
problcin sizes and timings: “Adopting these im- scientists frnm new areas try the algorithm without
provements, the approximate times for Fourier having to invest too n i d i time and energy
analysis are 10 minutes for 8 coefficients, 25 min-
utes for 16 coefficients, 60 minutes for 32 coefli-
cients, and 140 minutes for 64 ~ocfficicnts.”~This I t s effect
indicates a running time of about .37 Nlog Nmin- 1t:q difficult for me to overstate FFT‘s itnpor-
lites for an N-point DFI‘! tance. Much of its central place in digital sigual
Dcspitc thesc early discoveries of an Fm, it and image processing is due to the fact that it
wasn’t until James W. Cooley and John W. Tukey’s made working in the frequency domain equally
article that the algorithm gained any notice. l’he computati(inal1y feasible as working in tlie t e n -
story of their collaboration is an interesting one. poral or spatial domain. By providing a fast algo-
T i l c y arrived at the basic reduction while in a meet- rithm for convolution, the FFT cnablcd fast,
ing of President ICetitiedy’s Science Advisory C o n - large-integer and polynomial multiplication, as
mittee. Among thc topics discussed were tecliuiques well as efficient matrix-vcctor multiplication for
for offshore detection ofiiuclear tests in tlie Soviet Toeplitz, circulant, and other kinds of structured
Union. Ratification of a proposed Unitcd States- matrices. More generally, it plays a key role in
Soviet Union nuclear test haii depended on thc de- innst efficient sorts of filtcring algorithms. Modi-
vekipmem of a method to detect the tests withnut fications of the F I T arc one approach to fast al-
actually visiting Soviet nuclear facilitics. One idca gorithms for discrete cosine or sine transforms, as
was to analyze scismological tiine-series data ob- well as Chehyshev transforms. In particular, the
taincd from offshore seisniomcters, the length and discrete cosine transform is at the heart of MP3
number of which would reqnirc hst algorithm to encoding, which gives life to real-time audio
compute the DFT Other possible applications to streaming. Last hut not least, it’s also o m of the
national secnrity included the long-range acoustic few algorithms to make it into the movies-I can
detection of nuclear submarines. still recall tlie scene in No MGy Out where the im-
Richard Gamin of IBM was another participant agc-processing guru declares that he will need to
at this meeting, and when Tukcy showed him the “Fourier transform the image” to help Kevin
idea, he immediately saw a wide range of potential Costner sec the detail in a photograph!
applicability and qnickly set to getting the algorithm Even beyond these dircct technological appli-
implemented. H e was directed to Cooley, and, cations, the FbT influenced the direction of aca-
necding to hide the national security issncs, told dcmic rcsearch, too. ’Ihc FFT was one of the first
Coolcy that he wanted the code for another pro1)- instances of a less-than-straightforward algorithtn
Icm of interest: the &termination of tlie spin- with a high payoff in cfficiency used to compute
orientation pcriodicities in a 3D crystal of Hc3. something important. Furthermore, it raised the
Coolcy was iiivolved with other projects, and sat nahiral question, “Could an even faster algorithm
down to program the Cooley-Tukey FkT only after be found for the D F T ? ” (tlic answer is no7),
much prodding. In short order, lie and Tukey prc- thereby raising awareness of and heightening in-
pared a paper which, for a niathematics or computer terest in the subject of lowcr bounds and tlie
science paper, was published almost instantaneously analysis and devclopnient of efficient algorithms
(in six months).’ This publication, as well as Gar- in general. With respect to Shmucl Winograd’s

62 IN SCIENCE& ENGINEERING
COMPUTING
lower-bound analysis, Coolcy writes in the dis- search, and we can only wonder which of the many
cussion of the 1968 Arden I-Iousc Workshop on recent private technological discovcries might have
FFT, “These are the beginnings, I believe, of a prospered from a similar aniioimcenient.
branch of computer science which will prohably
uncover and evaluate other algorithms for high
speed computers.”8 The future FFT
Ironically, the F F T s prominence might have As torrents of digital data continue to swain into
slowed progress in other research areas. It pro- our computers, it s e e m thnt the FFT will continue
vidcd scientists with a big analytic hammer, and, to play a protniucnt role in our aiialysis and under-
for many, the world suddenly looked as thmigh it standing ofthis river of data. W h a t follows is a tirief
were hi11 of nails-even if this wasn’t always so. discussion of hihire FFT challenges, as well as a fcw
Researchers sometimes massaged problems that new directions ofrelated research.
might have hencfited from other, more appropriate
techniqncs into a D F r framework, siinply because Even bigger FFTs
the FF1‘was so efficient. Onc cxaiiiplc that comes Astronomy continues to be a chief consiiiiier of
to mind is some of the carly spectral-methods work large FIT tcclinology. T h e needs of projects like
to solve PDEs in spherical gcometly. In this case, iMAP (Microwavc Anisotropy Project) or LIGO
the spherical harmonics are a naniral set of basis (Laser InterFenitneter Gravitational-Wave Obscr-
functions. Discretization for nunicrical solutions vatory) require FETs of scvcral (even tens of) giga-
implies the coinpit“ti(in of discrete Lcgcndrc points. FFTs of this size do not fit in the iliain
transform (as well as FFl‘s). Many of the early m e m o r y oftnost machines, and these so-called out-
cotnputational approaches tried instcad to ail- ofcore FITSarc an active area ofresearch.9
proximate these expansions couiplctcly in terrns As cotiiputing techn~ilogyevolves, undoubtedly,
(if Fourier series, rather than address the develop- versions of the F i T will evolve to keep pace a n d
ment of an efficient I.egendre transform. take advantage of it. Different kinds of inenioty
Even now there arc still lessons to learn from rhc hierarchies and architectures present new chal-
F“s dcvclopInent. In this day and age, where any Icugcs and opporhiliitics.
new tcchnological idea s e e m fodder for Internet
vennirc capitalists and patent lawyers, it is natiiral Approximate and nonuniform FFTs
to ask, “Why didn’t IDM patent the lTT?” Coolcy For a variety of applications (such as fast MRI),
explained that because Tukey wasn’t an IBM e n - we need to compute DPTs for nonuiiiforinly
ploycc, IKM worried that it might not be able to spaced grid points and frequencies. Multipolc-
gain a patent. Consequently, IBM had a great in- hascd approaches efficiently compute thcsc quan-
terest i n putting the algorithni in the public do- tities in such a way that the miming time increases
mniti. T h e effect was that then nobody else could by a factor of
patcnt it either. This did not sceni like such a great
loss because at the time, the prevailing attihidc was
that a company made rnoncy in hardware, not soft-
ware. In fact, the FFl’vas designed as a tool to an- where t denotes the approximation’s precision.“’
alyze huge time series, in theory something only Algebraic approaches hasctl on efficient polyno-
supercomputers tacldctl. So, hy placing in the puh- inial evaluation are also possible.”
lic domain an algorithm that would make tinie-
series analysis feasible, more big companies might Group FFTs
have an interest in buying supercomputers (like The FFT might also he explained and interpreted
IBM mainframes) to do their work. using the language of group representation the-
Whether having thc FFT in the public domain ory--working along these lines raises some inter-
had the effcct IBM hoped for is nioot, h i t it cer- esting avenues for gcncralization, One approach is
tainly provided inany scientists with applications to view a 1D D I T oflcngth Nas computing the ex-
on which to apply the algorithm. T h e breadth of pansion of a function defined on C,”, the cyclic
scientific interests at the Ardeii workshop (held group of length N ( h e group of integers mod N) in
only two years after the paper’s publication) is truly ternis of the basis of irrcducihlc matrix elements of
impressive. In fact, the rapid pace of today’s tech- Clv,which arc precisely the familiar sampled expo-
nological dcvclopments is in iiiaiiy ways a testa- nentials: en(m)= e x p ( 2 x f i k m / N I . T h c FFT is a
ment to this open development’sadvantage. This is highly efficient algorithm for computing the cx-
a cautionary tale in today’s arena ofproprietary re- paiision in this hasis. More generally, a hioction on

]AN!JARY/FEBRUARY
2000 63
 any coinpact group (cyclic or not) 1x1san expati- References
sion in terms of a basis of irreducible matrix elc- I . E.O. Brigham, Thef~~tf~ou,ierTroniformondltrAppiicoli~onr, Pren-
menn (which generalize the cxponentials lroln the tie Hall Signal Processing Series, Englewood Cliffs, N.I., 1988.

point of view of group invariance). It's nahiral bi 2. M.T. Heideman, D.H. Ioiinran, and C.S. BUIIUI, "Gaurs and the
History of the Fait FourierTranrform," Anhive for Histo!yaf€mcl
wonder if efficicnt algorithms for perf(iriiiing this Scienter, Vol. 34, No. 3, 1985, pp. 261-277.
change of hasis exist. For example, thc problem of 3. F.Yater, "The Design and Analyriraf Factorial Experiments." l m ~
efficiently coniputing spherical harinonic expati- perial8ureou of SoiiSiiencer Tech. Comm., Vol. 35.1937.
sions falls into this kamework. 4. G.C. Danielion and C. Lanczor, "Some Improvementi in Practical
T h e first FFT for a tioncommutative finite Fourier Analysis and Their Application to X-Ray Scattering from
Liquids." I. fronkiin inst., Val. 233, Nor. 4 and S. 1942, pp.
group sccms to have h e m developcd by N a n 365-380 and 432452.
Willsky in the context of analyzing certain Mar-
kov processes.'* To date, fast algorithms exist for
. I. .
5. I.W. Coolevand I.W.Tukev. "An Aluoritimfor Machine Cdcula-
I

tion of Complex Fourier Series." MothemoticsofCompotation, Vol.

inany classcs of coinpact groups." Areas of ap- 19,Apr.. 1965, pp.297-301.

plications of this work include signal lirocessiiig, 6. I.W. Cooley, "The Re-Dircoveryof the Fast FourierTranrfarm Ai-
gorithm." Mikrochimica Acta, Vol. 3, 1987, pp. 3345.
data analysis, and robotics."
7. S. Winograd, "Arithmetic Complexityaf Computalionr," CBML
NSF Regional Canf. Series in Applied Mathemoticr, Vol. 33, SIAM,
Quantum FFTs Philadelphia, 1980.
Otic of the first great triuiiiphs of thc qnan- 8. "Special lirue an Fast FourierTranrfarm and Its Application to
hnll-computing nlodel is Peter Shori fast algo- Digital Filtering and Spectral Analysis," I€€€ Trans. Audio €kclron-
irr, AU-15, No. 2, 1969.
rithm for integer factorimtion on a quantum
9. T.H. Carmen and D.M. Nicol, "Periarming Out-of-Core FFTr on
c ~ m p u t c r .At
' ~the heart of Shor's algorithm is a Parallel Dirk Systems." Porallei Campuling, Val. 24, No. 1, 1998,
subroutine that compntcs (on a qumtuni c o n - pp. 5-20.
puter) the DFT of a hinary vector representing 10. A. Dutt and V. Rakhlin, "Fast Fourier Transforms for Nonequi-
an integer. T h e itiyilementation of this trans- spaced Data." SIAMI. Scimt,fic Computing, Vol. 14, No. 6, 1993,
pp. 1368-1 393; continued in Appiied and Computotional Hor-
forin as a sequcllce of one- and two-hit quanhim moni~AnoIysis,VoI.2, No. 1, 1995, pp. 85-100.
pates, now called the qiiantum FFI; is effectively 11. D.K. Marlen and D.N. Rockmore, "Generalized FFTr-A Suwey
the Cooky-Tukcy FIX rcalized as a particular of Some Recent Recults." Croups aodComputatioon, il,DIMACS Ser,
factorization of the Fourier matrix into a product Dimete Math. Theoret. Camput. Sci., Vol. 28, Amer. Math. Soc.,
Providence, R.I., 1997, pp. 183-237.
of matrices composed as certaio tensor products
12. A.S. Wilisky, "On thenlgebraic Struclure of Certain Partially Ob-
of two-by-two uiiitaly matrices, each of which is rewzblc Finite~StateMarkov Procerrer." loformotion ond Control,
a so-called local unitary transform. Similarly, Vol.38,1978,pp.179-212.
thc quanhmi soluti(m to thc Modified Dentscli- 1 3 . D.N. Rockmare, "Some Applications of Generalized FFTr (An Ap-
Josza problem uses the matrix factorization aris- pendix with D. Healy)." CrouprandCamputoti~onli,DiMACSSeriex
on Discrete Moth. Theoret. Comput. Sci., Val. 28, American Math-
ing kom Yates's algorith~ii.'~ Extensions of these ematical Society, Providence, R.I., 1997, pp. 329-369.
ideas to the more general group transforms 14. P.W. Shar, "Polynomial-Time Algorithms for Prime Factorization
mentioned earlicr are currciitly Iicing explorcd. and Diicste Logarithms on a Quantum Computer," SIAM]. Com-
puting,Vol. 26, No.S.1997, pp. 1484-1109.
15. D. Simon, "On the Power of Quantum Computation." Proc. 35th
Annual ACM Symp. on Foundations of Computer Science, ACM
hat's the FIT-hotli parent aiid Prerr,NewYarh,I994,pp.l16~123.
child of the digital revolution, a
computational technique at tlic Daniel N. Rockmore is an associate professor of math-
nexus of tlic worlds of business and ematics and computer science at Dartmouth College,
entertainment, national security and public coiii- where he also serves as vice chair of the Department of
tnunication. Ntliongli it's anyone's guess as to Mathematics. His general research interests are in the
what lies over tlic next horizon in digital signal theory and application of computational aspects of
processing, the FFT will most likely bc in the group representations, particularly to FFT generaliza-
thick of it. 3 tions. He received his BA and PhD in mathematics from
Princeton University and Harvard University, respectively.
Acknowledgment In 1995, he was one of 15 scientists to receive a five-
Special thanks to lim Cooley, Shmuel Winograd, and year NSF Presidential Faculty Fellowship from the White
Mark Taylor for helpful conversations. The Santa Fe Ins- House. He is a member of the American Mathematical
titute provided partial support and a very friendly and Society, the IEEE, and SIAM. Contact him at the Dept.
stimulating environment in which to write this paper. NSFof Mathematics, Bradley Hall, Dartmouth College,
Presidential Faculty Fellowship DMS-9553 7 34 supported Hanover, NH 03755; rockmore@cs.dartmouth.edu;
part of this work. www.cs.darthmouth.edu/-rockmore.

64 COMPUTING & ENGINEEUINC

I N SCIENCE