Computational Bioacoustics: Machine Learning for Animal Acoustics

Computational bioacoustics is the interdisciplinary field that applies digital signal processing, acoustic feature extraction, and machine learning to record, detect, classify, and model non-human biological sounds. Spanning terrestrial forests, freshwater streams, and deep ocean basins, computational bioacoustics enables researchers to monitor biodiversity, estimate animal population density, and study social communication across massive acoustic datasets that cannot be reviewed manually.

Over the past decade, the discipline has transitioned rapidly from manual spectrogram reading and handcrafted acoustic metrics to deep neural networks, self-supervised acoustic encoders, and generative sequence modeling.

The Computational Bioacoustics Pipeline

A modern automated bioacoustic analysis pipeline progresses through four distinct computational stages:

[ Raw Audio Waveform (Time Domain) ]
                  │
                  ▼
 [ Feature Representation & Preprocessing ]
 ├── Short-Time Fourier Transform (STFT)
 ├── Mel-Scale Log-Spectrograms (2D time-frequency)
 └── Learned Waveform Convolutions (1D SincNet / Codec Encoders)
                  │
                  ▼
   [ Neural Architecture & Representation ]
 ├── 2D Convolutional Neural Networks (ResNet, EfficientNet, MobileNet)
 ├── Self-Supervised Transformers (HuBERT, AVES, wav2vec 2.0)
 └── Neural Codec Tokenizers (SoundStream, RVQ) + Autoregressive LM
                  │
                  ▼
     [ Downstream Task & Evaluation ]
 ├── Species / Caller Presence Detection
 ├── Multi-label Classification & Confidence Scoring
 └── Unsupervised Vocal Motif Clustering & Sequence Modeling

Acoustic Feature Extraction

Before machine learning models can classify animal vocalizations, continuous acoustic pressure waves must be transformed into computational representations:

1. Short-Time Fourier Transforms (STFT): Splitting waveforms into short windowed frames (e.g., Hann or Hamming windows of 25–50 ms) to compute discrete Fourier spectra over time.
2. Mel-Scale Filterbanks: In terrestrial bioacoustics, linear frequencies are often warped onto the logarithmic Mel scale to mimic mammalian auditory resolution. In marine bioacoustics, however, linear or custom ultrasonic frequency bands are preferred because delphinid biosonar operates between 40 kHz and 130+ kHz.
3. Learned 1D Convolutions: Modern self-supervised architectures frequently bypass fixed-window spectrograms entirely, feeding raw waveforms directly into learned 1D convolutional filterbanks that optimize time-frequency resolution end-to-end.

Core Machine Learning Architectures

1. Convolutional Neural Networks on Spectrograms

The canonical baseline for automated bioacoustic classification is treating 2D audio spectrograms as images. The leading example in terrestrial bioacoustics is BirdNET, developed jointly by the Cornell Lab of Ornithology and the Chemnitz University of Technology:

Evaluating over 984 bird species across North America and Europe, BirdNET demonstrated that residual convolutional networks trained on 3-second spectrogram patches could reliably classify species even in high-noise outdoor environments. Code and models are publicly accessible in the BirdNET-Analyzer repository.

2. Self-Supervised Bioacoustic Encoders (AVES)

Supervised learning requires thousands of manually verified audio labels—a major bottleneck in marine and tropical ecology. Self-supervised learning (SSL) overcomes this by pre-training transformer models on vast unlabeled audio archives using masked prediction tasks.

The Earth Species Project introduced AVES (Animal Vocalization Encoder based on Self-Supervision), adapting the HuBERT speech architecture specifically for non-human bioacoustics:

Trained across multi-species bioacoustic datasets, AVES generates dense acoustic embeddings that significantly outperform general speech representations on downstream animal sound classification. The AVES codebase and model weights are open-sourced under the MIT License in the earthspecies/aves repository.

3. Neural Codecs and Autoregressive Sequence Transformers

For complex delphinid vocalizations, the DolphinGemma research project pairs Google's SoundStream neural audio codec with an autoregressive sequence transformer (~400M parameters):

Using Residual Vector Quantization (RVQ), SoundStream compresses continuous hydrophone audio into discrete token streams. The transformer models sequential transition probabilities across tokens:

P(T_1, T_2, ..., T_N) = ∏ P(T_i | T_1, ..., T_{i-1})

This allows researchers to quantify vocal entropy, discover recurring acoustic motifs, and analyze temporal order without human labeling bias.

Evaluation Standards in Bioacoustic Machine Learning

Evaluating machine learning in ecological settings differs fundamentally from standard benchmark datasets:

• Class Imbalance & Rare Species: Target animal vocalizations may constitute less than 0.1% of a continuous 24-hour recording. Evaluating models using standard accuracy or Receiver Operating Characteristic (ROC-AUC) yields misleadingly optimistic results. Researchers prioritize Precision-Recall Area Under the Curve (PR-AUC) and strict False Positive Rate (FPR) thresholds.
• Out-of-Distribution Noise: Rain, wind, snapping shrimp, boat engine cavitation, and passing storms produce acoustic interference that can trigger false detections if training data lacks background noise diversity.
• Label Scarcity: Few species have verified annotations. Zero-shot and few-shot bioacoustic classification benchmarks (such as BEANS) are essential for measuring real-world generalization.