“In the realm of enchanted LLMs, GPT-4’s magic recently saved a dog’s life by detecting a disease through symptoms. Now, scientists blend animal wisdom with LaMDA’s secrets, embracing GPT-3’s essence. Within the Earth Species Project, they pursue the holy grail: the decoding of animal tongues. Past knowledge weaves a tapestry, birthing a melodic ‘Rosetta Stone,’ revealing creatures’ secret symphonies.”

The shape of the problem

While it seems natural to allow NLP algorithms to find the structure of animal communication, the problem has many more facets. One of the most basic pitfalls is that noises are simply just one of the modalities that animals use to communicate. Visual and tactile stimuli play an equally important role in animal communication as auditory stimuli.

Human communication is built on many social norms and structures, which serves as a fallback when it comes to decoding ancient languages. However, animal communication has no such reference point, as each species has their own syntax of communication. 

Collecting animal data also comes with its own set of ethical problems. Research has found that even if experiments aren’t constructed to be manipulative and are purely observational, scientists still intervene to monitor. This may cause distress to the animals they are studying, poisoning the data and raising ethical concerns that scientists are now being required to evaluate.

Apart from decoding the communication itself, making sense of animal data is also a behemoth undertaking. As opposed to human data, which is annotated in an intuitive way (for humans), animal data is not only difficult to procure, but also needs specific research to annotate. 

These are just some of the issues that researchers need to face when looking at animal communication. In fact, scientists have been trying to decode animal communication since the late 1950s, resulting in a comprehensive corpus of research on the topic.

Building on this legacy, the Earth Species Project (ESP) aims to solve some of the field’s long-standing issues through the use of AI. From mapping out the vocals of crows to building a benchmark of animal sounds, ESP is laying the foundation for future AI research in the field. Towards this end, the project also has a roadmap detailing how they wish to use AI to shorten the communication gap between species. 

Solution: SSL

According to ESP, there are 4 main facets to the communication problem — data, fundamentals, decode, and communicate. The project aims to bring AI solutions for each of these sub-problems. For data, it is building out self-supervised models to annotate and interpret collected data, among other undertakings. On the fundamental side, it is focused on building foundational models, such as the AVES (animal vocalisation encode based on self-supervision) model. 

For decode, ESP is building out self-supervised pattern detection models. Finally, for communication, it is creating a generative AI solution with the help of Google to generate the communicative signals of animals. To this end, one of their most interesting projects currently is the generative vocalisation experiment, which adds to researchers’ arsenal of bioacoustics research tools. Using this approach, researchers can play back edited vocalizations, thus deepening their understanding of animal communication. 

Using AI, ESP was able to turn semantic relationships into geometric relationships — the same process that was used to make NLP algorithms more efficient in the past. By visualising the relationships between various words, it becomes possible to create a shape for a given language. This has been successfully implemented for human languages, but ESP believes they can go further. Aza Raskin, one of the co-founders of the Earth Species project, stated, 

“We’re building on top of the research in the human domain, we’re developing models that can work with bats and whales. With humans you almost always know you can ground truth at some point, you can always check. That’s not the case with animals.”

However, it is possible for certain self-supervising algorithms to approach this problem from a different angle. By finding patterns in a large number of similar datasets, latent space created by LLMs can provide important insights into animal communication. But, there is a risk of solving the problem of communicating with animals without knowing what we are communicating to them. 
The idea of picking self-supervised learning (SSL) to solve this problem is also an interesting choice. SSL algorithms are arguably the least explainable form of AI, but they might just be the best fit for understanding animal vocalisations. Using the fundamentals set in place today, ESP believes that they can “build an audio chatbot for animal communication that nobody yet understands.”