Spectrograms in Bioacoustics: Interpreting Time-Frequency Audio Representations

A spectrogram is a visual time-frequency representation that displays how the spectral energy of an acoustic signal changes across time. In computational bioacoustics, spectrograms represent the indispensable visual and mathematical bridge between continuous sound waveforms and biological signal analysis.

Whether inspecting delphinid signature whistles, classifying bird species via convolutional neural networks, or tuning the visualizer parameters in the DolphinGemma Spectrogram Tool, interpreting spectrograms requires an understanding of the Short-Time Fourier Transform (STFT), windowing functions, and the fundamental physical limits of acoustic resolution.

Mathematical Foundations: The Short-Time Fourier Transform

Continuous time-domain waveforms x(t) recorded by hydrophones represent total acoustic pressure over time, but obscure individual frequency components. Because animal vocalizations are non-stationary (their frequency content constantly changes), standard global Fourier transforms cannot show when specific frequencies occurred.

The Short-Time Fourier Transform (STFT) resolves this by multiplying the signal by a sliding analysis window w(t) centered at time τ:

X(τ, ω) = ∫ x(t) · w(t - τ) · e^(-iωt) dt

The continuous spectrogram S(τ, ω) is the squared magnitude (power spectrum) of the STFT:

S(τ, ω) = |X(τ, ω)|²

In digital systems, the Discrete Short-Time Fourier Transform computes the Discrete Fourier Transform (DFT) over sequential frames of length N samples, shifted by a hop size of H samples:

Continuous Waveform:  [─── Frame 1 ───]
                             [─── Frame 2 ───]
                                  [─── Frame 3 ───]
                                   ◄── Hop Size H ──►
                             ◄────── Window Length N ──────►

Windowing Functions and Spectral Leakage

If an arbitrary segment of an audio waveform is truncated abruptly without a smooth taper (a rectangular window), the sharp discontinuity at the frame edges introduces artificial high-frequency energy across all frequency bins—an artifact known as spectral leakage.

To minimize leakage, bioacousticians apply smooth windowing functions:

| Window Function | Main-Lobe Width | Sidelobe Attenuation | Bioacoustic Recommendation | | :--- | :--- | :--- | :--- | | Hann (Hanning) | Moderate (4π / N) | -31.5 dB | Excellent general baseline for whistle contours | | Hamming | Moderate (4π / N) | -42.7 dB | Good for speech and low-frequency baleen calls | | Blackman-Harris | Broad (8π / N) | -92.0 dB | Best for high dynamic range & weak harmonic detection | | Rectangular | Narrow (2π / N) | -13.3 dB | Severely distorted by leakage; avoid in bioacoustics |

The Time-Frequency Trade-Off: Narrowband vs. Wideband Spectrograms

Choosing window length (N) and FFT size (N_FFT) dictates the resolution characteristics of the resulting spectrogram:

Narrowband Spectrogram (Large Window, e.g., N = 2048 at 96 kHz):
  Δt ≈ 21.3 ms (Poor time resolution; clicks smear horizontally)
  Δf ≈ 46.9 Hz (Superb frequency resolution; resolves closely spaced whistle harmonics)

Wideband Spectrogram (Small Window, e.g., N = 256 at 96 kHz):
  Δt ≈ 2.6 ms (High time resolution; individual clicks resolved cleanly)
  Δf ≈ 375 Hz (Coarse frequency resolution; harmonic contours merge into thick bands)

As demonstrated by Lammers, Au, and Herzing (2003), dolphin whistles and burst pulses frequently overlap in the same recording. A bioacoustician must often render dual spectrograms—one wideband view to measure inter-click intervals of burst pulses, and one narrowband view to trace frequency modulation contours of signature whistles.

Common Spectrogram Artifacts and Misinterpretations

When interpreting bioacoustic spectrograms, field researchers must recognize non-biological artifacts:

1. Preamplifier Overload (Harmonic Clipping)

When a vocalizing dolphin approaches very close to a hydrophone, acoustic pressure can exceed the hydrophone's dynamic range. This saturates the preamplifier, squaring off the waveform:

• Visual Artifact: Spurious, perfectly integer harmonics extending all the way to the Nyquist frequency.
• Verification: Inspect the raw time-domain waveform. If wave peaks are flattened (clipped), the upper harmonics are hardware artifacts, not biological vocal features.

2. Surface Multipath Reverberation (Echo Trails)

Because seawater and air have a severe acoustic impedance contrast, the ocean surface acts almost like an acoustic mirror (reflection coefficient R ≈ -1).

• Visual Artifact: A whistle or click appears doubled, accompanied by a delayed, phase-inverted shadow contour.
• Verification: Calculate surface reflection travel time based on hydrophone deployment depth and sound speed (c ≈ 1,500 m/s), as formulated by Medwin (1975):

3. Window Scalloping Loss

If an acoustic signal's true frequency falls midway between two discrete FFT bins, its energy splits between adjacent bins, artificially reducing peak amplitude by up to 1.4 dB (with a Hann window). Zero-padding the FFT (N_FFT > N) interpolates the spectrum smoothly and restores peak location accuracy.

Implications for Deep Learning and Computer Vision

Modern machine learning models (such as BirdNET, ResNet classifiers, and Mel-spectrogram transformers) process spectrograms as 2D numerical matrices:

• Logarithmic Compression: Raw power values (|X|²) span many orders of magnitude. Converting to decibels (dBFS = 10 · log₁₀(S)) normalizes contrast and ensures low-amplitude vocalizations are not obscured by dominant noise floors.
• Fixed Window Constraints: Standard CNNs assume fixed-size image patches (e.g., 3-second windows with fixed FFT bins). If input sample rates or FFT parameters differ between training and field deployment, convolutional kernels fail to recognize known vocal contours.
• Self-Supervised Alternatives: Modern foundation audio codecs (such as SoundStream) bypass rigid STFT hyperparameter choices by learning convolutional time-frequency representations directly from raw hydrophone waveforms.