DARCY : Design and development of an underwAteR event deteCtion sYstem based on embedded AI

The Lab-STICC is a research unit of the French national center for scientific research (CNRS) involving two universities (Université de Bretagne Occidentale, Brest; Université de Bretagne Sud, Lorient) and three graduate schools of engineering (IMT Atlantique, ENSTA Bretagne and ENIB). Under the title UMR 6285, it is attached to department INS2I and INSIS of the CNRS.

Context

Underwater equipment such as drones play a crucial role in marine research, monitoring, and resource management. However, their deployment is hindered by significant constraints, including limited communication capabilities, energy consumption, and computational challenges. Previously we have focused on developing algorithms for underwater fish detection [1, 2] and fish species classification [3, 4]. The objective of this internship is to design and develop a prototype underwater event detection system using embedded artificial intelligence. 

Underwater event detection can encompass a wide range of scenarios. For example, a sudden change in luminosity could trigger an image enhancement algorithm, while the appearance of a single fish in the camera’s field of view might activate a species classification algorithm. Similarly, the detection of a group of fish could initiate a behavioral analysis algorithm. Other events of interest include the detection of marine debris and suspended particles, which are critical for environmental monitoring.

This approach involves running event detection algorithms directly on the embedded system, circumventing communication bottlenecks. Given the high computational demands of event detection algorithms, selecting the right algorithm requires careful consideration of computing resources and energy consumption beside their precision. For instance, during long missions  with strict battery constraints, maintaining a monitoring with 24 frames per second (fps) rates may not be feasible, requiring the use of adaptive event detection intervals.

Various embedded platforms exist with specific support for AI applications. In this internship, we consider three low-power consumption platforms: NVIDIA Jetson Nano, Raspberry PI 5 Model B, and Beaglebone AI. Two types of operating systems are considered:  Linux-based OS and small footprint real-time OS with more robust scheduling algorithms [5] such as Zephyr or FreeRTOS.

Experiments will be carried out on publicly available datasets such as Fish Recognition Ground-Truth dataset, LifeClef 2015 Fish dataset, TrashCan dataset and Trash-ICRA dataset.

Main activities

1. Develop an event detection algorithm for underwater images

- Perform a survey on the state of art of event detection algorithms in the computer vision domain.

- Develop an event detection algorithm for underwater images. The architectures such as YOLO or Tiny-YOLO shall be considered.

- Carry on a training to create a custom model optimized for underwater event detection.

- Test the developed algorithm to ensure accurate detection in underwater environments.

2. Deployment and evaluation on embedded platforms

- Deploy the  algorithms on three embedded platforms with a Linux-based OS: NVIDIA Jetson Nano, Raspberry PI 5 Model B, and Beaglebone AI

- Comparison between the platforms by the following metrics: power consumption, memory consumption, performance analysis

- Study deployment of the algorithm with a small-footprint real-time operating system such as Zephyr RTOS

Remuneration: Standard internship support for public establishments in France (~624 €/month)

Duration: 6 months

Host institutions: Lab-STICC teams OSE & SHAKER, Université de Bretagne Occidentale (UBO) and Ecole Nationale d'Ingénieurs de Brest (ENIB). The internship is financed by ISblue - Interdisciplinary Graduate School (https://isblue.fr/)

References

[1] Ben Tamou, A. "Reconnaissance d’espèces de poissons dans des images vidéo sous-marines".  thèse de doctorat, 2021.
[2] Ben Tamou, A., Benzinou, A., Nasreddine, K. "Multi-stream fish detection in unconstrained underwater videos by the fusion of two convolutional neural network detectors." Applied Intelligence 51.8 (2021): 5809-5821.
[3] Ben Tamou, A., Benzinou, A., Nasreddine, K. "Live fish species classification in underwater images by using convolutional neural networks based on incremental learning with knowledge distillation loss." Machine Learning and Knowledge Extraction 4.3 (2022): 753-767.
[4] Ben Tamou, A., Benzinou, A., Nasreddine, K. "Targeted data augmentation and hierarchical classification with deep learning for fish species identification in underwater images." Journal of imaging 8.8 (2022): 214.
[5] Le Boudec, A., Singhoff, F., Tran, H. N., Rubini, S., Levieux, S., & Skrzyniarz, A. (2023, December). Work-In-Progress: Could Tensorflow applications benefit from a mixed-criticality approach?. In 2023 IEEE Real-Time Systems Symposium (RTSS) (pp. 427-430).

Master 2 or Engineer internship. It is preferable for the candidate to have a background or a first-time experience in one of the two domains: (1) Artificial Intelligence, (2) Embedded systems. A strong background in software engineering is well appreciated. An intermediate-advanced level of English proficiency is required.