Bemjax

What’s new in whale speech decoding

/

Sperm whales talk in clicks. Over the last two years the way researchers represent that talk has changed from a flat list of labels into a factorised, compositional code — and the tools doing the factorising are the same generative models you already know. Here is where coda research stands, for a reader who knows some ML and nothing about whales.

1 · What a coda is, and what the old representation was

Sperm whales emit two kinds of click: echolocation clicks for foraging sonar, and codas — short stereotyped sequences of 3–40 clicks, under two seconds long, exchanged in duet-like back-and-forth or in group chorus before diving and while socialising. They are never produced alone. That single constraint is the reason to treat a coda as communication rather than a by-product of sensing.

Codas are culturally learned, not genetic. Whales segregate into vocal clans of hundreds or thousands that identify themselves by their coda repertoire; each clan dialect carries at least twenty coda types. What a whale “says” is decided by the clan it grows up in.

Until 2024 the working representation was a flat categorical label set — roughly 20–40 “coda types” named by their inter-click-interval pattern (5R = five regular clicks; 1+1+3). A hand-built vocabulary of discrete labels laid over a continuous acoustic signal. Everything below is the story of replacing that label set with something factorised.

The corpus is what makes it possible. The Dominica Sperm Whale Project has followed the same identified families off Dominica since 2005. Project CETI added fixed hydrophone arrays, clingfish-inspired suction-cup bio-logging tags (three synchronised hydrophones plus GPS, depth, temperature, light and motion), drones and gliders — about 10,000 coda recordings over two decades. The recordings did not exist before the instrument did.

2 · 2024: from a label set to a factorised code

Sharma et al., Nature Communications 2024. Instead of asking which of N labels is this coda, ask what dimensions generate it. Four features — and the split between them is the actual finding:

  • rhythm — the pattern of inter-click intervals — context-independent
  • tempo — overall duration / click rate — context-independent
  • rubato — smooth variation of duration across successive codas in an exchange — context-sensitive
  • ornamentation — an extra click appended to a coda within a run — context-sensitive

Rhythm × tempo cross into a combinatorial base; rubato and ornamentation modulate it according to the exchange the coda sits inside. The resulting inventory is nearly an order of magnitude larger than the coda-type list it replaces — proposed as a sperm whale phonetic alphabet, the first phonetic alphabet proposed for a non-human species.

For an ML reader this is exactly the move from a categorical codebook to a factorised, compositional code with two context-invariant factors and two conditioned on the surrounding sequence. The context-dependence of half the factors is what makes it look like a system with structure rather than a fixed repertoire of calls.

3 · 2025: an orthogonal spectral axis (vowels)

Beguš et al., Open Mind (MIT Press) 2025. Everything above reads timing. This reads the spectrum, and finds two recurrent, discrete coda-level spectral patterns — the a-coda vowel and the i-coda vowel — plus transitions between them inside a single coda, i.e. diphthongs.

The load-bearing claim: these spectral properties combine freely with the timing features and are independent of coda type. Same rhythm, different vowel. The code has a whole axis nobody had been measuring.

The human analogy is stated precisely: click count and click timing correspond to vowel duration and pitch; the spectral properties of the clicks correspond to formants. Mechanistically it is source–filter — the phonic lips vibrate as the source (as vocal folds do) and the distal air sac is hypothesised to act as the filter (as the vocal tract does), shaping the resonances that separate one coda vowel from another.

One terminology trap worth flagging: a sperm whale click is not the click consonant of human phonetics (which is non-pulmonic and turbulent). It is better understood as a glottal pulse — the kind of thing that builds a vowel. Confusing the two poisons every analogy downstream. The paper also argues the pattern is structured and discretely distributed rather than a physical artefact of the whale’s movement.

4 · What the model says is there, and what nobody is claiming

The methodological problem is that there is no ground truth. You cannot ask the whale and you cannot label a held-out set. So the 2025 work inverts the question: train a generative model on raw audio, then interrogate what it treats as informative.

CDEV — causal disentanglement with extreme values (Beguš, Leban & Gero, Royal Society Open Science 2025). Train on raw audio, then drive individual latent variables far past the range seen in training, and use causal inference to measure which observable properties of the output actually move. The architecture is fiwGAN, an InfoGAN adaptation of WaveGAN that splits its input into incompressible noise and a featural code, so the code can afterwards be read as what the network found worth encoding. Training set: 2,209 samples across the five commonest coda types.

Four properties came out as information-carrying: number of clicks, regularity of click timing, spectral mean, acoustic regularity. Two were already hypothesised by biologists; two were not. That the model recovers the known two is what licenses belief in the new two. Generalisation check: the network generated codas resembling the 9R type, which was never in its training data.

WhAM — Whale Acoustics Model, NeurIPS 2025. The first transformer that generates synthetic codas from any audio prompt, built by finetuning VampNet — a masked acoustic-token model pretrained on music — on the 10k coda corpus. Its learned representations classify rhythm, social unit and vowel well despite being trained only to generate. Code is open at github.com/Project-CETI/wham.

And the unglamorous enabling piece: the first automatic coda detector and annotator (Gubnitsky et al., Scientific Reports 2025), built on graph-based clustering that exploits the expected similarity between the clicks within one coda. It works at low SNR, separates codas from echolocation clicks, and pulls apart codas from whales calling simultaneously.

Not claimed: no semantics, no translation, no ecological or behavioural causation — the CDEV paper says so in as many words. This is a phonology-scale result: an inventory and its combinatorics. It says the signal has more structure than the old label set could hold. It does not say what any of it means.

The whole region as a graph — Project CETI, its people, funders, instrument, field site and the research programme, 98 nodes laid out left-to-right: open the interactive diagram.

Sources (the four papers, plus the detector and WhAM):