Introduction
Sound Synthesis
Mathematics and Music
College of Engineering, Science and Environment, University of Newcastle, Australia
BMath Seminar
September 7, 2023
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Table of Contents
1 Introduction
2 Sound Synthesis
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
What is sound
What is sound?
Vibrations transmitted through an elastic solid or liquid or gas, with frequencies in the
approximate range of 20 to 20,000 hertz, capable of being detected by human organs
of hearing.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Mathematics and Music
”There is geometry in the humming of the strings, there is music in the spacing of
the spheres.” — Pythagoras
1 Basic applications: Counting, rhythm, scales, intervals, patterns, symbols,
harmonies, time signatures, overtones, tone, pitch.
2 Advanced applications: Sound synthesis, sound recognition, sound separation,
music transcription, note recognition. There are sound to MIDI (Musical
Instruments Digital Interface) converters but they are still not really accurate.
3 Age detection through the sound and face.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Music is periodic variation in air pressure
• P = A sin(2* 𝜋 *f*t), P is pressure in decibels or pascals and t is time in
seconds
• A is the amplitude in pascals or decibels
• f is the pitch or frequency in Hertz (Hz) (1 cycle per second)
• Period is given by 1/f
Mathematics and Music
• Mathematics deals with the structure, order, and relation that has evolved
from counting, measuring, and describing the shapes of objects.
• When a few notes are played together, or the same note is repeated a few
times on an instrument, rhythms are created.
• Mathematics is useful to study the structure, order and relation of rhythms
and notes.
• Pythagoras made a fundamental contribution to the connection between
mathematics and music.
Pythagoras (569-475 BC)
• Experimenting with the tones produced when plucking strings of
different lengths, he found that some specific ratios of string lengths
created pleasing combinations (“harmonies”) and others did not.
• He discovered that a string exactly half the length of another will play
a pitch that is exactly an octave higher (double the pitch) when struck
or plucked. Thus, he identified the note intervals, or distances
between notes, that form the primary harmonic system which is still
used today.
Harmonics: Modes of vibration of strings are called harmonics
Pythagoras
• Pitches of notes in music are based on proportional relationships.
• The lowest frequency produced by the oscillation of an object is called ‘the
fundamental frequency’. Including integer multiples of this fundamental
frequency makes ‘overtone series’.
• If the lowest or slowest frequency (often called the fundamental
frequency) is 100 Hz, then the overtones would be 200 Hz, 300 Hz and so
forth. (The overtone series is often referred to as harmonics.)
• Greek octave is based on his findings.
Pythagoras
• Pythagoras discovered the overtone series and thus the ratio 2:1 of
pitches forming an octave.
• This mathematical finding is universal. This 2:1 ratio to form an octave
is the basis of all documented musical systems although musical
cultures around the world are diverse.
• Pythagoras thus observed the intrinsic properties of mathematics and
physics within music. This connection might explain why many
physicists and mathematicians are also musicians.
Einstein
A quotation from Einstein: “The theory of relativity occurred to me by
intuition, and music was the driving force behind that intuition….My
new discovery was the result of musical perception” (Suzuki, 1969, 90).
88 Keys Piano
• 7 Octaves (Each has 12 keys)
• 3 keys on the left and 1 key on the right
• These 88 keys have different pitches (frequencies)
• 3 keys in the left have frequencies: 27.5, 29.13524 and 30.86771
• The keys are numbered from 1 to 88 from the left to the right
• For the nth key the frequency is given by the formula f(n)= 440 ∗ 2("#$%)/()
• These keys have also scientific names: A0,A#0,B0 for the first three.
• Then C1, C#1, D1, D#1, E1, F1, F#1, G1, G#1, A1, A#1, B1 for the first octave and so on.
Piano keys with frequencies and names
Introduction
Sound Synthesis
The first line of ’Mary had a little lamb’
The following matlab code creates the first line of the popular song: Mary had a
little lamb.
fre=[329.6280 293.6650 261.6260 293.6650 0 329.6280 329.6280 .
329.6280 0 293.6650 293.6650 293.6650 0 329.6280 391.9950 3
f = @(f0,t) exp(-0.05*t) .* sin(2*pi*f0*t);
n = 10000; %number of sampling points for a half second note
t = [0:n]/(2*n); %play a note only a half second
song = [];
for k = 1:length(fre)
song= [song f(fre(k),t)];
end
sound(song,2*n)
Mary
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Additive Synthesis: Fourier Series
By combining sine waves of different frequencies and amplitudes, we can create
new sounds. These are often called timbres.
The lowest frequency in a note is called the fundamental frequency.
A music instrument generates a tone which can be approximated by many sine
waves.
A tone having many frequencies has rich, complex tones. For example,
f (x) = a1 sin(πx) + a2 sin(2πx) + a3 sin(3πx) + a4 sin(4πx).
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Fourier Series
A sufficiently smooth function f (x) is expanded in terms of sin and cos as
∞
a0 X nπ nπ
f (x) = + an cos x + bn sin x .
2 n=1
L L
The coefficients an and bn are given by the Euler’s formulas:
1 L
Z nπ
(A) an = f (x) cos x dx for n ≥ 0.
L −L L
1 L
Z nπ
(B) bn = f (x) sin x dx for n ≥ 1.
L −L L
These coefficients an , bn are called Fourier series coefficients of the function f (x).
Convergence theory?
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Fourier Series: Sawtooth-like Wave
Find the Fourier series of
f (x) = x for − π < x < π, f (x + 2π) = f (x), x−π ∈
/ 2πZ.
In this case, the Fourier coefficients are given by
1 π
Z
an = x cos(nx) dx = 0, n ≥ 0.
π −π
Z π
1 2 (−1)n+1
bn = x sin(nx) dx = − cos(nπ) = 2 , n ≥ 1.
π −π n n
It can be proved that the Fourier series converges to f (x) at every point x where f is
differentiable, and therefore:
∞
a0 X
f (x) = + [an cos (nx) + bn sin (nx)]
2 n=1
∞
X (−1)n+1
= 2 sin(nx), for x−π ∈
/ 2πZ.
n=1
n
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Synthesisers
Synthesisers create different sounds by combining some simple sounds
(waveforms) to create more complex sounds. Some simple sound
waves: sine, square, triangle and sawtooth waves.
Introduction
Sound Synthesis
Fourier Series: Example for Partial Sums
Consider the partial sums of Fourier series:
a0 a0 π π
S0 (x) = , S1 (x) = + a1 cos x + b1 sin x , ··· ,
2 2 L L
N
a0 X nπ nπ
SN (x) = + an cos x + bn sin x .
2 n=1
L L
N =1 N =2
N =5 N = 25
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Fourier series: Square Wave
Consider a square wave f (x) of period 2L. The following square wave is an odd function. So
a0 = 0, and an = 0 for all n ∈ N, and
2
Z L nπx
bn = f (x) sin( ) dx
L 0 L
2
=− (cos(nπ) − 1).
nπ
Hence
2 4
[1 − (−1)n ] =
bn = if n is odd,
(nπ) (nπ)
and bn = 0 if n is even. The Fourier series for L = 1 is, therefore,
∞ ∞
4 X 1 4 X 1
f (x) = sin (nπx) = sin ((2n − 1)πx)
π n=1,3,5,... n π n=1 2n − 1
N =1 N =2 N =5 N = 25
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Additive Synthesis: Fourier Series
By combining sine waves of different frequencies and amplitudes, we can create new
sounds. These are often called timbres. The lowest frequency in a note is called the
fundamental frequency. A music instrument generates a tone which can be
approximated by many sine waves. A tone having many frequencies have rich,
complex tones.
Mary Additive
Mary Simple
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Subtractive Synthesis: Fast Fourier Transform (FFT)
Subtractive synthesis is based on modifying the spectrum of the original signal.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Subtractive Synthesis: Sound Comparison
Many techniques in sound synthesis are based on modifying the spectrum of a
synthetic signal to match with a spectrum of an organic sound.
Original music using sawtooth wave
After using subtractive synthesis
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Sound Synthesis with Optimisation
We want to break sound into smaller pieces. Since sound waves are periodic signals,
we can express using sine and cosine waves resulting in a Fourier series. Let
y : [0, T ] → R be a continuous signal. For example, we can approximate a sound
signal y by using sin and cos like
n
!2
Z T X
ya = min y(t) − (ai cos(wi t) + bi sin(wi t)) dt
0 i=1
1 This can be digitised by sampling the signal y and also the sum
P n
i=1 (ai cos(wi t) + bi sin(wi t)) at a finite number of time points.
2 The optimisation problem is nonlinear if we minimise with respect to wi .
3 We can use some information of the sound signal like physical properties to
simplify wi .
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Optimisation Bottleneck
If we approximate the global signal (a 1/2 second A4 note looks like below), we
do not capture the local property of the signal accurately.
If we approximate the the signal locally (using around 5-6 milliseconds note), we
lose the accuracy globally.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Frequency Modulation Synthesis
The main idea behind Frequency Modulation is that we modulate the frequency over
time resulting in change in frequency over time. A simple frequency modulation is
obtained by setting
F (t) = sin(2πfc t + I sin(2πfm t)),
where fc is called the carrier frequency, fm is called the modulation frequency and I
is called the index of modulation, which is also the strength of modulation or the
intensity of the modulation.
We can include more terms to model a wide variety of sounds:
F (t) = sin(2πfc t + I1 sin(2πfm1 t) + I2 sin(2πfm2 t)).
Yamaha’s First Synthesiser DX-7 is based on this technique.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Frequency Modulation
Using fc = 1, fm = 2 and I = 12 in the modulated signal
F (t) = sin(2πfc t + I sin(2πfm t)),
we have the following result. Here the carrier signal is
Fc (t) = sin(2πfc t).
Synthetic bass simulation using the frequency modulation:
Mary Frequency Modulation
Bishnu P. Lamichhane,
[email protected] Mathematics and Music
Introduction
Sound Synthesis
Frequency Modulation
Fourier series for the frequency modulation:
sin(fc t + I sin(fm t)) =
P∞
J0 (I) sin(fc t) + k=1 Jk (I) sin((fc + kfm )t) + (−1)k sin((fc − kfm )t) ,
where Jk is a Bessel function (first kind) of order k. The function Jk is a solution to
the following Bessel differential equation
d2 y dy
x2 + x2 − k 2 y = 0
2
+x
dx dx
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Bessel Function
A summation formula for Jk (x) is
∞
X (−1)m x 2m+k
Jk (x) = .
m=0
m!Γ(m + k + 1) 2
A more useful form in the frequency modulation for integer values of n is
1 π
Z Z π
1
Jn (x) = cos(nτ − x sin τ ) dτ = ℜ ei(x sin τ −nτ ) dτ,
π 0 2π −π
√
where i = −1 is the imaginary number, and ℜ stands for the real part of the
expression.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Wavetable Synthesis
Wavetable synthesis produces a sound signal by cycling through a table of values that
represent a waveform.The wavetable method generates different frequencies of a
signal by changing the speed with which it cycles through table entries.
1 Wavetable synthesis is based on creating periodic waveforms by employing
arbitrary periodic waveforms.
2 The output waveform changes in time as one wavetable is mixed with another,
creating a changing waveform.
3 Used in many commercial synthesisers: Waldorf, Ableton, PPG Wave, etc.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Physical Modelling Synthesis
This sound synthesis method is based on generating sound by using a mathematical
model. Modelling a physical source of sound from a musical instrument like piano or
flute leads to differential equations, which are then approximated by some numerical
techniques.
1 This technique can produce a wide variety of sounds: the sound may be often
too refined or too good.
2 It requires not much memory compared to digital instruments based on sound
recording or other techniques.
3 Commercial packages: IK Multimedia’s Modo Bass and Modo Drum, Modartt
Pianoteq and Organteq, Yamaha used some ideas in Yamaha VL1 in 1994.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Some Interesting Facts
Human hearing range: 20 Hz to 20 kHz.
Children can hear a slightly wider range of frequencies around 15 Hz to 25 kHz.
For middle aged person, the upper limit is often 14 kHz.
Children voice frequencies range from 250 to 400 Hz
Adult female voice around 165 to 255 Hz
Adult male voice around 85 to 155 Hz
Bishnu P. Lamichhane, [email protected] Mathematics and Music
Introduction
Sound Synthesis
Final Slide
Play Three Instruments
Thank you. Questions.
Bishnu P. Lamichhane, [email protected] Mathematics and Music
