AI-dentifying whales in the Arctic
Will machine learning technology designed to detect marine mammals scale up the quality and quantity of marine mammal data?
As people around the world celebrate World Whale Day later this month, artificial intelligence (AI) is quietly working behind the scenes to support whale research. One project, called Flukebook, is tasked with identifying individual whales and dolphins based on photos taken of their flukes (tails). Another called Project CETI hopes to use machine learning to "understand what sperm whales are saying, and hopefully also talk back to them." In Canada, Möbius, developed by Whale Seeker, is searching through 100,000 aerial images for whales and other marine mammals in the Arctic for Fisheries and Oceans Canada (DFO).
Without AI, "it would take maybe six months to a year for one person to go through 50,000 images," says Charry Tissier, CEO and co-founder of Whale Seeker. By using Möbius to do the job in a fraction of the time, DFO hopes to free up expertise, expand monitoring and other marine mammal projects, and include the most up-to-date science in decision-making.
A large pod of narwhals detected by Möbius, Whale Seeker’s AI-powered image annotation system. The polygonal annotations suggest the whales are oriented in one direction and possibly travelling (Credit: Fisheries and Oceans Canada)
Teaching Möbius to spot a whale
Möbius is a type of machine learning known as computer vision, in which the AI is 'taught' to identify objects, places, people, animals, and other elements. The process of teaching an AI to recognise elements begins by creating a large dataset of images, which are annotated with the name of the item present in the image. For example, an image of a whale would be annotated 'whale,' and an image of sea ice would be annotated 'sea ice'. The AI uses this annotated collection of images, known as a training dataset, to learn and recognise different elements.
The quality of the training dataset plays a crucial role in how accurately an AI can recognise what is in an image. Möbius's training dataset was carefully chosen to represent the types of images typically gathered by surveyors; "aerial photographs, taken with a normal, very high resolution off the shelf camera that's either payload on a drone, or strapped to the belly of an aeroplane," Charry Tissier explains.
Then there is the annotation itself, something which Whale Seeker takes very seriously.
Computers to do the grunt work
If the images in the training dataset aren't correctly annotated, the AI will be more likely to make identification mistakes. "AI isn't magic," says Charry Tissier. "We need good data going in, in order for there to be good data coming out." To ensure high-quality annotation, Whale Seeker only works with marine mammal identification experts for their annotation.
Alongside quality, the size of the training dataset also plays a crucial role in recognition accuracy. Every new dataset that Möbius will look at will have its own nuances, such as ocean or weather conditions on the day the images were taken or the camera's altitude. For each new dataset, “we pick out a subset of images and have those annotated by a human expert,” says Charry Tissier. Those new annotated images are then added to the initial training dataset for Möbius to relearn. Similarly, human experts are brought back in when Möbius is less confident about what is in an image.
In this ‘humans-in-the-loop’ approach, "humans are being used for their expertise, and the computers are doing what computers do best, which is doing the grunt work of going through a bunch of data," Charry Tissier says. “We're aiming to be as good as an expert human observer — but more consistent and faster.”
Whale Seeker works with organisations targeting conservation including the Whale and Dolphins Conservation.
Read more about the community scientists that are monitoring ocean and coastal environments in the Marine Professional article 'Citizen science - the power of the people'.
To become a member of the Marine Mammals SIG, log into your My IMarEST account, click on My Special Interest Groups and then tick the boxes of the SIGs you’d like to join. You can then also join the group on Nexus, our networking platform.”
Dr Sam Andrews is a marine ecologist and science writer