Short-time Fourier transform

A spectrogram (using the Short-Time Fourier Transform, STFT) is a visual representation of how the frequencies of a signal change over time. It works by breaking the signal into small time segments (windows), applying the Fourier Transform to each, and displaying the resulting frequencies as a heatmap, where time is on the x-axis, frequency on the y-axis, and color intensity represents amplitude. A longer window gives you finer frequency resolution ( $Δ f \approx 1 / T$ ) but poorer time localization; a shorter window does the opposite. This is related to uncertainty principle.

Our brain-ear system performs a process similar to a spectrogram. The cochlea in the inner ear acts like a frequency analyzer, breaking incoming sounds into different frequency components, much like the Short-Time Fourier Transform (STFT). These frequency signals are then sent to the brain, which processes and interprets them as speech, music, or other sounds.

What the time-domain signal is: the STFT gives the frequency content (what the cochlea extracts), but the original function of "the sound" — its height over time — is literally the displacement of the speaker membrane (equivalently the instantaneous air pressure at a point). A higher peak = greater membrane displacement = louder. How fast it oscillates = frequency = pitch. So the Y-axis of the raw signal is membrane displacement, while the STFT/cochlea re-describes that same signal as a frequency spectrum. In a compact disc, each sample is the value of that displacement (pressure) at one instant; the D/A converter positions the membrane there, and a reconstruction filter smooths the steps.

Gabor transform: something similar but with a kind of Gaussian window... It optimizes for the uncertainty principle.
wavelet transform: we choose a window size depending on the frequency:

High frequencies → very short windows (good time resolution)
Low frequencies → long windows (good frequency resolution)