COFUND PhD position – Detection, Recognition, and Tracking of Marine Species Using Video Analysis

Title of the thesis project: Detection, Recognition, and Tracking of Marine Species Using Video Analysis: Toward Dynamic Population Estimation for Biodiversity Conservation

Scientific description of the research project

Background

Tracking the evolution of marine populations is a key indicator for assessing the impacts of human activities and climate change on marine ecosystems. Traditional monitoring methods, such as manual

surveys or sonar-based sampling, often face limitations in terms of spatial and temporal coverage, cost,

and intrusiveness. Video analysis, combined with artificial intelligence (AI), presents a transformative approach for automated and large-scale monitoring of marine species.

Research Question

How can video analysis, supported by artificial intelligence algorithms, be used to detect, recognize, and track marine species to estimate their populations and monitor temporal trends?

Objectives

The objectives of this thesis will be:

1. Develop AI models capable of accurately detecting and recognizing various marine species in complex underwater environments (e.g., turbid waters, dynamic backgrounds, …).
2. Create robust tracking techniques to monitor individuals and groups across video sequences and avoid redundant counts.
3. Assess population changes through spatial and temporal data collected via underwater cameras.
4. Identify the key scientific and technological challenges that must be addressed to ensure widespread adoption of these methods.

Previous works

1. As part of our collaboration with the New Caledonian company VISIOON, we developed tracking algorithms designed to detect and track fish species. These algorithms were applied to videos captured by 360° underwater cameras placed on the seabed at depths ranging from 5 to 10 meters. However, the scope of this work was limited to relatively simple scenarios: the cameras were positioned in shallow, calm, and clear waters, and the videos contained a limited number of species to simplify the evaluation of detection and tracking challenges.

The developed approaches enabled the semi-automatic processing of video by:

• Identifying relevant sequences in the videos (moments when species appear in the camera's field of view);
• Detecting and tracking all moving objects over time and across space;
• Determining the optimal moments for potential identification of each object.

Each identification was indexed to create a comprehensive database of the species present at the studied site. However, the recognition task has not been developed yet.

It should be noted that these approaches were carried out without using artificial intelligence techniques.

2. Poh Lee Wong's thesis, *Automated Fish Detection and Identification* (2015), focused on monitoring fish to analyse their behaviour and identify signs of illness. The study involved analysing video sequences of small shoals of fish (10 to 20 individuals). However, the experiments were conducted in aquariums under controlled lighting, with the fish filmed from above.

3. Frédéric Petit's thesis, *Processing and Analysis of Underwater Colour Images: Physical Models and Quaternionic Representation* (2010), addressed the challenge of colour correction in underwater images to counteract light absorption and scattering effects. The developed algorithms restored the colours to resemble their appearance "out of the water," effectively reducing or eliminating the bluish and greenish tints typically seen in underwater imagery.

This prior research provides valuable insights and tools for addressing the challenges of this thesis.

Scientific Challenges

The goal of this thesis is to enhance the robustness of detection methods, enabling their application across a wide range of scenarios. Additionally, it seeks to integrate a recognition phase. To achieve these objectives, the thesis will tackle the following challenges:

1. Detection in Complex Environmental Conditions

• Visual Noise: Underwater environments are often characterized by poor visibility due to turbidity, low light levels, and particles in suspension. These factors degrade video quality and pose significant challenges for image processing and species detection algorithms. Models need to be trained to distinguish species from background noise under such conditions.
• Dynamic Backgrounds: The ocean is a constantly changing environment, with moving water, light refraction, and shifting vegetation. These dynamic elements can interfere with detection techniques by confusing them with the target species.
• Morphological Variability: Many marine species exhibit significant variability in appearance due to factors such as growth stages, sexual dimorphism, or environmental influences. Algorithms must be robust enough to handle these variations while avoiding misclassification.

2. Species Recognition and Classification

• Lack of Comprehensive Datasets: Annotated underwater datasets are limited, particularly for rare, cryptic, or poorly studied species. Developing diverse and high-quality datasets is a critical challenge for effective training of AI models.
• Bias in Classification: Unbalanced datasets may lead to over-representation of common species, reducing accuracy when identifying less frequently encountered ones. Addressing this requires strategies like data augmentation or transfer learning.

3. Accurate Tracking and Counting

• Individual Differentiation: Identifying and tracking individual organisms in a dense population is complex, particularly when individuals are visually similar. Models must incorporate features such as size, movement patterns, and unique markers to distinguish individuals.
• Preventing Double Counting: In sequences where organisms reappear after moving out of frame, systems must avoid counting the same individuals’ multiple times. In our case we will use a 360° camera, but when working with "unfolded" images, a fish disappearing on the right, will reappear on the left. Advanced object re-identification techniques will be needed.
• Behavioural Challenges: Many marine animals have rapid and unpredictable movement patterns, making traditional tracking algorithms insufficient.

4. Temporal and Spatial Analysis

• Wide-Area Monitoring: To assess trends over large marine areas, algorithms must integrate data from multiple cameras and platforms. This requires consistent normalization and calibration of inputs to ensure compatibility.
• Trend Analysis: Building models to detect subtle population trends over time requires accurate, long-term data collection and computational techniques for extrapolation under varying environmental conditions.

Methodology

To address the issues described above, the work will take place in several stages.

1. Data Acquisition: The first step involves gathering extensive video data from diverse underwater environments. In this research we will used fixed cameras. The main idea is to deploy stationary cameras in targeted locations such as coral reefs, seagrass beds, and fish aggregating sites to capture long-term footage. However, we retain the possibility to used remotely operated vehicles (ROVs) and underwater drones to explore areas that are difficult to access, such as dynamic coastal zones.

The collected video data will cover different environmental conditions, including clear and turbid waters, varying depths, and diverse biodiversity hotspots. This comprehensive dataset will serve as the foundation for algorithm training and testing.

Visioon already has many videos taken near coral reefs. More than 100 hours of video are already available. It will carry out new acquisition campaigns by varying the sites (seagrass beds for example) and the conditions (turbidity, depth, swell, weather).

An annotation phase will then be necessary in order to create a data set that can be used for the learning phases.

The core of the research work will focus on developing AI-based algorithms, first, for detecting marine species and then tracking them.

2. Species Detection and Recognition: The objective is to implement computer vision models using deep learning architectures to accurately identify and classify marine species in video footage. These models will be trained on labelled datasets to ensure their ability to differentiate between closely related species. A key challenge will be designing models robust enough to handle underwater visual complexities, including turbidity, variable lighting conditions, and dynamic backgrounds.

Particular attention will be given to evaluating whether frugal models, with a manageable number of parameters, can effectively handle detection and recognition tasks.

3. Multi-Object Tracking: Detection and recognition alone are insufficient for accurately censusing (counting) species. The analysis must incorporate multi-object tracking (MOT) algorithms to follow individual organisms across consecutive video frames. These approaches will need to address challenges such as occlusion (when species are temporarily obscured by objects) and re-identification (ensuring the same individual is not counted multiple times after leaving and re-entering the frame). To enhance tracking accuracy, we will explore integrating behavioural models, movement trajectories, and distinctive visual features.

4. Validation: To ensure the reliability and accuracy of the developed system, a validation phase will compare the algorithm's results with ground truth data annotated by experts and findings from existing literature.

The creation of ground truth will be conducted in collaboration with marine biologists, who will label species and behaviours in selected video segments, providing a robust reference for evaluation. Performance metrics will be employed to assess the effectiveness of detection, recognition, and tracking tasks.

A continuous improvement process will be implemented, using validation results to iteratively refine the models. This process will address errors such as false positives (incorrect identifications) and missed detections, enhancing the overall performance of the system.

Visioon will provide its spid-up software to help carry out the annotation phase. It will also contribute to the annotation phase by calling upon its experts. This validation process will ensure that the system meets high standards of scientific accuracy and robustness.

5. Case Study: The final phase will involve deploying the developed system at pilot sites to assess

its practical effectiveness and real-world applicability. To conduct real-world testing, the selection of test sites will focus on ecologically significant and representative sites, such as marine protected areas (MPAs), coral reefs or deep-sea ecosystems. If time permits, the analysis could also explore short-term population dynamics, with the aim of identifying emerging trends and patterns. The case study will demonstrate the framework's potential for long-term monitoring and its capacity to generate actionable insights for marine conservation.

Expected Outcomes

1. Automated Monitoring Framework: A fully functional system for detecting, recognizing, and tracking marine species in diverse underwater environments.

2. Comprehensive Datasets: High-quality annotated datasets from various marine ecosystems, serving as a resource for future research.

3. Population Insights: Detailed analyses of marine species population trends over time, supporting biodiversity assessments and conservation strategies.

4. Enhanced Conservation Efforts: A scalable, non-intrusive tool that can be deployed globally to monitor and protect marine ecosystems.

Significance

This research lies at the intersection of marine biology, artificial intelligence, and quantitative ecology, offering opportunities for interdisciplinary collaboration. By addressing the challenges outlined, this project could pave the way for real advances in marine biodiversity monitoring and conservation.

Since its creation in 1993, La Rochelle University has been on a path of differentiation.

Thirty years later, as the university landscape recomposes itself, it continues to assert an original proposition, based on a strong identity and bold projects, in a human-scale establishment located in an exceptional setting.

Anchored in a region with highly distinctive coastal features, La Rochelle University has turned this singularity into a veritable signature, in the service of a new model. Its research it addresses
the societal challenges related to Smart Urban Coastal Sustainability (SmUCS).

The new recruit will join the Laboratory of Informatics Image Interaction (L3I).

Cotuelle: University of Science and Technology of Hanoi (USTH), Vietnam. ICT Lab.

Master's degree