Skip to content

Time-Frequency Analysis Explorer

Copy this iframe to your website:

1
<iframe src="https://dmccreary.github.io/signal-processing/sims/time-frequency-analysis-explorer/main.html" width="100%" height="607px" scrolling="no"></iframe>

Run MicroSim in Fullscreen

Edit in the p5.js Editor

Description

A single Fourier transform says which frequencies a signal contains but not when they occur. The short-time Fourier transform (STFT) answers both questions by transforming one short, windowed frame after another:

\[X[m,k] = \sum_{n=0}^{N-1} x[n + mH]\,w[n]\,e^{-j2\pi kn/N}\]

Frame \(m\) holds \(N\) samples starting at sample \(mH\), where \(H\) is the hop size, multiplied by a window \(w[n]\). A spectrogram is a picture of \(\lvert X[m,k]\rvert\) with time across, frequency upward and colour for level.

The signal is one second long, sampled at \(f_s = 8000\) Hz. The display has three plots:

  1. Waveform with a red time cursor. The shaded band marks the \(N\) samples of the frame at the cursor.
  2. Spectrogram. Each frame is drawn at the centre of its \(N\) samples and each bin at its centre frequency, so every coloured rectangle is one value of \(X[m,k]\). The grey strips at the two ends are times for which no complete window fits.
  3. Spectrum at the cursor, drawn sideways so that it shares the frequency axis of the spectrogram. The dashed line marks the frequency the signal really has at that instant, and the colour bar underneath is the key to the spectrogram colours.

Choose the Signal and the Window type from the menus, set the Window length N (32 to 1024 samples) and the Overlap (0 to 90%) with the sliders, and pick the Colour map and Scale (magnitude in dB, linear magnitude or linear power). Click or drag in the waveform or the spectrogram to move the cursor. This MicroSim supports the section on spectral analysis in Chapter 7: DFT, FFT and Frequency Domain Analysis.

Mathematical Concepts

The trade-off. A window of \(N\) samples spans a time \(\Delta t = N/f_s\) and gives bins spaced \(\Delta f = f_s/N\) apart, so

\[\Delta t \times \Delta f = 1\]

whatever \(N\) is. A long window separates close frequencies but smears anything that changes during the window. A short window follows fast changes but gives wide frequency bins. This is the uncertainty principle of time-frequency analysis: both resolutions cannot be made fine together.

Window length \(N\) 32 64 128 256 512 1024
Time span \(\Delta t\) (ms) 4 8 16 32 64 128
Bin spacing \(\Delta f\) (Hz) 250 125 62.5 31.25 15.63 7.81

Overlap sets the hop size, \(H = N(1 - \text{overlap})\) rounded to a whole number of samples, and the number of frames is \(\lfloor (8000 - N)/H \rfloor + 1\). More overlap gives more frames and a smoother picture along the time axis, but it does not improve the resolution: that is fixed by \(N\).

Level. Amplitudes are scaled so that a steady sinusoid of amplitude 1 centred on a bin reads 0 dB, and the dB scale spans 80 dB (from \(-80\) dB to 0 dB). The windows are the periodic (DFT-even) forms, so a tone centred on a bin occupies exactly three bins with the Hann window.

Signals. The chirp sweeps linearly from 500 to 3500 Hz, so its frequency is \(500 + 3000\,t\) Hz. The frequency-shift-keyed (FSK) signal sends 16 bits, each 62.5 ms long, as a 1 kHz or a 2 kHz tone. The note sequence plays C5, E5, G5, C6 and G5, each 200 ms long, as a plucked tone with three harmonics. All three are synthesized by the MicroSim; no recorded audio is used.

How to Use

  1. Click or drag in the waveform or the spectrogram to move the time cursor.
  2. Move the Window length slider and watch the cells of the spectrogram change shape: tall and narrow for a short window, flat and wide for a long one.
  3. Move the Overlap slider and read the hop size and the number of frames in the box at the upper right.
  4. Point at the spectrogram to read the time, frequency and level of the cell under the mouse.
  5. Change the Window, the Colour map and the Scale.

Lesson Plan

Learning Objectives

After using this MicroSim, students will be able to:

  1. Explain why a spectrogram is needed for a signal whose frequency content changes with time.
  2. Compute the time span, bin spacing and hop size of an STFT from the window length, sampling rate and overlap.
  3. Analyze how the window length trades time resolution against frequency resolution.
  4. Choose STFT settings that suit a given signal.

Target Audience

Undergraduate students in a digital signal processing course. Plan 30 minutes for the activities below.

Prerequisites

  • The DFT, bin spacing and window functions
  • Spectral leakage
  • Sampling rate and sample period

Activities

Activity 1: What a spectrogram adds (5 min). Choose the chirp, the Hanning window, \(N = 256\) and 50% overlap. Drag the cursor from the left to the right of the spectrogram.

  • The dashed line in the side panel rises with the cursor. At the frame centred at \(t = 0.496\) s it reads 1988 Hz, which is \(500 + 3000 \times 0.496\).
  • The peak of the spectrum at the cursor sits on that line: its strongest bin is 2000 Hz, within half a bin (15.6 Hz) of the true value.
  • A single DFT of the whole second would show energy between 500 and 3500 Hz but nothing about when each frequency occurs.

Activity 2: The resolution trade-off (10 min). Keep the chirp and the Hanning window and step \(N\) from 32 to 1024.

  • Check the banner against the table above: \(\Delta t\) doubles and \(\Delta f\) halves at each step, and their product is always 1.
  • The ridge is thickest at \(N = 32\) (about 800 Hz wide, measured 20 dB below its peak) and thins as \(N\) grows. It is thinnest at \(N = 256\) (about 150 Hz), then widens again: about 175 Hz at \(N = 512\) and about 310 Hz at \(N = 1024\).
  • The chirp rises 3000 Hz per second, so during a 128 ms window it moves \(3000 \times 0.128 = 384\) Hz. The bins are narrow at \(N = 1024\), but the window is too long for the frequency to hold still. The best compromise is near the \(N\) at which the Hanning main lobe (\(4f_s/N\)) equals the sweep during the window (\(3000N/f_s\)), which is \(N \approx 292\).
  • With \(N = 256\), switch between the rectangular and Hanning windows on the dB scale. At 40 dB below the peak the Hanning ridge is about 260 Hz wide but the rectangular ridge is about 2100 Hz wide: its slowly decaying sidelobes show as a haze around the ridge.

Activity 3: Bits and notes (10 min). Use the Hanning window and 50% overlap.

  • Choose the FSK signal with \(N = 64\). Each bit lasts 62.5 ms, the window only 8 ms, so the bit edges are sharp. Each tone is a band of three bins (375 Hz) at 1000 or 2000 Hz.
  • Change to \(N = 1024\). Each tone is now a thin line, but the 128 ms window is longer than two bits, so the edges are blurred and in 8 of the 14 frames both tones are above \(-20\) dB at once. At \(N = 64\) only 6 of the 249 frames show both. Which \(N\) would you use to read the bits?
  • Choose the note sequence. Put the cursor at \(t \approx 0.24\) s (E5, 659 Hz) and at \(t \approx 0.44\) s (G5, 784 Hz). With \(N = 32\) both notes have the same strongest bin, 750 Hz. From \(N = 64\) upward they fall in different bins, and at \(N = 256\) the strongest bins are 656 Hz and 781 Hz, within 3 Hz of the true values.
  • At \(N = 1024\) find the harmonics of one note: G5 shows peaks near 784, 1568 and 2352 Hz.

Activity 4: What overlap does (5 min). With the chirp and \(N = 256\) change the overlap from 0% to 90%.

  • At 0% the hop is 256 samples (32 ms) and there are 31 frames. At 50% it is 128 samples and 61 frames. At 90% it is 26 samples (3.25 ms) and 298 frames.
  • The picture becomes smoother along the time axis, but the thickness of the ridge does not change. Overlap fills in the picture; only \(N\) and the window type change the resolution.

Assessment

  1. For \(N = 512\) at \(f_s = 8000\) Hz, what are the time span and the bin spacing? \(\Delta t = 512/8000 = 64\) ms and \(\Delta f = 8000/512 = 15.6\) Hz; the product is 1.
  2. For \(N = 128\) and 75% overlap, what are the hop size and the number of frames in one second of signal? \(H = 128 \times 0.25 = 32\) samples (4 ms) and \(\lfloor (8000 - 128)/32 \rfloor + 1 = 247\) frames.
  3. Why does no window length give sharp time resolution and sharp frequency resolution together? Because \(\Delta t \times \Delta f = 1\): shortening the window to localize an event in time widens the bins by the same factor.
  4. For the chirp, why is the ridge thinner at \(N = 256\) than at \(N = 1024\) even though the bins are four times narrower at \(N = 1024\)? The frequency moves 384 Hz during the 128 ms window, which smears the ridge by more than the narrow bins save.
  5. Which window length would you choose to read the bits of the FSK signal, and which to measure the pitch of the notes? \(N = 64\) to \(128\) for the bits (8 to 16 ms against a 62.5 ms bit, and the tones are 8 to 16 bins apart); \(N = 256\) or more for the notes, where the bins are 31 Hz apart or less and the strongest bin lies within half a bin (16 Hz) of the pitch.

Common Misconceptions

  • A longer window always gives a better spectrogram. At \(N = 1024\) the chirp ridge is about twice as thick as at \(N = 256\) and the FSK bit edges are blurred over about 128 ms. Longer windows trade time resolution for frequency resolution.
  • Increasing the overlap improves the resolution. Move the Overlap slider: the hop and the number of frames change but the ridge keeps its thickness, because the resolution is set by \(N\) and the window.
  • The spectrogram shows the exact frequency at each instant. Compare the peak of the spectrum at the cursor with the dashed line: the peak is a bin centre, and each coloured cell stands for an interval of \(\Delta t\) and \(\Delta f\).

References

  1. Short-time Fourier transform - Wikipedia - definition of the STFT, its frames, window and hop size
  2. Spectrogram - Wikipedia - how the squared magnitude of the STFT is displayed against time and frequency
  3. Window function - Wikipedia - Hann, Hamming, Blackman and rectangular windows and their sidelobes
  4. Chirp - Wikipedia - the linear sweep used as the first test signal
  5. Frequency-shift keying - Wikipedia - the two-tone digital signal used as the second test signal
  6. The Scientist and Engineer's Guide to Digital Signal Processing - Steven W. Smith - free online textbook with chapters on the DFT and on windows

Further reading (print): J. B. Allen and L. R. Rabiner, "A unified approach to short-time Fourier analysis and synthesis," Proceedings of the IEEE, 65(11), 1558-1564, 1977; A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing, 3rd ed., Pearson, 2010, Chapter 10 (time-dependent Fourier analysis).