Shine white light through a prism and it fans out into a rainbow : the prism reveals that “white” was secretly a mixture of every colour. The Fourier transform is a prism for signals. It takes a sound, an image, a stock price, any wiggly function of time, and reveals the hidden mixture of pure tones it is made of : how much of each frequency is present. This one idea underpins audio compression, image formats, quantum mechanics and, as we will see, the way Transformers encode the position of a word. But it raises a nagging question : of all possible building blocks, why should sine waves be the right ones ? This post answers that, keeping the intuition simple and the mathematics honest.
1. The big idea : signals are recipes of sine waves
A sine wave $\sin(2\pi f t)$ is the purest oscillation there is : a single, unwavering frequency $f$. The claim at the heart of Fourier analysis is startling in its generality.
In plain words : just as a chord is several pure notes played together, any repeating waveform, however jagged, is really a stack of simple sine waves added up. Fourier analysis hands you the sheet music.
2. Why sinusoids, and not something else ?
There are two deep reasons sine waves are the privileged ingredients.
- They are “orthogonal”. Over one period, the average product of two sine waves of different whole-number frequencies is exactly zero. They do not interfere. This means we can measure “how much of frequency $n$ is present” without the other frequencies contaminating the answer, each ingredient can be extracted cleanly.
- They are the natural modes of physics. Sinusoids are the shapes that pass through any stable, linear, time-invariant system (a vibrating string, an electrical circuit, an echoing room) unchanged except for a rescaling. Feed in a pure tone and you get out the same tone, louder or softer and shifted, never a different frequency. No other family of functions has this property.
In plain words : sine waves are chosen not by taste but by mathematics, they are the only building blocks that stay out of each other’s way and that nature itself prefers.
3. The Fourier series
For a signal $f(t)$ that repeats with period $T$ (base frequency $\omega = 2\pi/T$), the recipe is written
\[\begin{equation} f(t) = a_0 + \sum_{n=1}^{\infty}\Big( a_n \cos(n\omega t) + b_n \sin(n\omega t) \Big). \label{eq:series} \end{equation}\]The amplitudes are recovered by the orthogonality trick above : to find how much of frequency $n$ is present, multiply the signal by that pure wave and average over a period (the contamination from all other frequencies integrates to zero) :
\[\begin{equation} a_n = \frac{2}{T}\int_{0}^{T} f(t)\cos(n\omega t)\,dt, \qquad b_n = \frac{2}{T}\int_{0}^{T} f(t)\sin(n\omega t)\,dt. \label{eq:coeffs} \end{equation}\]In plain words : each integral in $\eqref{eq:coeffs}$ asks “how much does the signal resemble this particular pure wave ?” , it is a similarity score, exactly the kind of projection a dot product performs, but for functions.
4. Watching a square wave being built
Nothing makes $\eqref{eq:series}$ more convincing than watching it work. A square wave, all sharp corners and flat tops, seems the least “sine-like” shape imaginable 1. Yet it is just
\[f(t) = \frac{4}{\pi}\left( \sin t + \frac{\sin 3t}{3} + \frac{\sin 5t}{5} + \frac{\sin 7t}{7} + \cdots \right),\]odd harmonics only, each a little weaker than the last. The animation adds them one at a time : with a single sine it is a gentle wave, and as more harmonics join, the corners sharpen and the tops flatten toward the ideal square (grey).
5. From series to transform
The Fourier series handles periodic signals. For a signal that does not repeat, we let the period $T \to \infty$, the discrete list of harmonics blurs into a continuous range of frequencies, and the sum becomes an integral. Using Euler’s identity $e^{i\theta} = \cos\theta + i\sin\theta$ to package sine and cosine together, we arrive at the Fourier transform :
\[\begin{equation} \hat{f}(\xi) = \int_{-\infty}^{\infty} f(t)\, e^{-2\pi i \xi t}\, dt. \label{eq:ft} \end{equation}\]In plain words : $\hat{f}(\xi)$ is still just “how much of frequency $\xi$ is in the signal”, the same similarity score as $\eqref{eq:coeffs}$, now measured for every frequency on a continuum. The transform is invertible : $f$ and $\hat f$ are two views of the same object, one in the time domain, one in the frequency domain. A sharp spike in $\hat f$ means a strong pure tone ; a spread-out $\hat f$ means a rich mixture.
6. Why it matters
Once a signal is in the frequency domain, things that were hard become easy. Removing hiss from audio is just deleting high-frequency components ; JPEG and MP3 compression throw away the frequencies your eyes and ears barely notice ; solving certain differential equations turns into simple algebra. The workhorse that makes this practical is the Fast Fourier Transform (FFT) (Cooley & Tukey, 1965), which computes $\eqref{eq:ft}$ for $N$ samples in $N\log N$ operations instead of $N^2$, one of the most consequential algorithms ever written.
And there is a charming connection back to deep learning. The Transformer tags each word position with a vector of sines and cosines of many different frequencies, exactly the ingredients of a Fourier series. Fast-oscillating components pin down fine position, slow ones the coarse position, so the whole vector is a Fourier-style “fingerprint” of where a word sits, the same principle that lets $\eqref{eq:coeffs}$ distinguish one frequency from another.
7. Conclusion
The Fourier transform is a change of perspective : stop describing a signal by what it does at each moment, and start describing it by which frequencies it is made of. Sine waves earn their central role honestly, they are the only ingredients that stay out of each other’s way and that linear physics leaves intact, so the recipe they define is unique and meaningful. From a prism splitting light to a Transformer locating a word, the same idea keeps reappearing : complicated things are often simple mixtures of pure oscillations, if only you look at them through the right prism.
References
- Cooley, J. W., & Tukey, J. W. (1965). An Algorithm for the Machine Calculation of Complex Fourier Series. Mathematics of Computation, 19(90), 297–301.
@article{Cooley1965, author = {Cooley, James W. and Tukey, John W.}, title = {An Algorithm for the Machine Calculation of Complex Fourier Series}, journal = {Mathematics of Computation}, volume = {19}, number = {90}, pages = {297--301}, year = {1965} }
-
Joseph Fourier introduced these series in 1822 while studying heat flow, to considerable scepticism, his contemporaries doubted that sums of smooth sines could represent functions with sharp corners. The Gibbs overshoot in Figure 1 is a trace of exactly that tension. ↩