MICRO : A Mixed criticality scheduling approach for Implementation of CNN applications on Zephyr RTOS

The Lab-STICC is a research unit of the French national center for scientific research (CNRS) involving two universities (Université de Bretagne Occidentale, Brest; Université de Bretagne Sud, Lorient) and three graduate schools of engineering (IMT Atlantique, ENSTA Bretagne and ENIB). Under the title UMR 6285, it is attached to department INS2I and INSIS of the CNRS.

Context 
The internship aims to study the application of mixed criticality scheduling (MCS) in the implementation of systems in which several tasks with different criticality levels coexist. We consider non-critical tasks that perform opportunistic treatment, represented by convolutional neural networks (CNN), as well as critical tasks with hard timing constraints. We are particularly interested in the "anytime" aspect, in which the quality of the results gradually increases with the computation time. The goal is to achieve a "proof of concept" (PoC) with two elements below:

- Implementation of an MCS scheduler in a real-time operating system (RTOS). The RTOS targeted in this project is Zephyr [1].
- Implementation of CNN processing in “anytime” mode on embedded platforms. The PoC is used to study and compare the MCS approach on the Zephyr RTOS with classical approaches such as embedded Linux

Problem statement
The main problem tackled in this internship is the uncertainty related to the application of mixed-criticality scheduling analysis to embedded systems with CNN applications. The uncertainty comes from two aspects:

- Lack of implementation/support of a mixed-criticality scheduler in RTOS: Although there has been a significant theoretical research effort in this area, the only existing implementation is a plugin of the LitmusRT benchmark [4], which does not meet our requirement to build a real functional “proof of concept” on an embedded platform.
- Lack of experimental results in the literature: The complexity of deploying CNN applications on an RTOS with a small footprint kernel and limited support for external libraries is also a challenge. In addition, the use of several external libraries makes the application of classical static timing analysis techniques impractical. We focus on a measurement-based approach in this project. Finally, real-time operating systems are not considered in the literature as part of scheduling problems, even though they can introduce significant overheads.

For CNN applications, we focus on using U-Net [2] for image segmentation, YOLO [3] for object detection and their variants optimized for embedded systems. Regarding RTOS, we focus on Zephyr - a small footprint kernel based RTOS. The motivation to use Zephyr in this project, besides its open source nature, is based on two reasons: a wide support of embedded platforms (500+) and an active development community.

Objective
The main objective is to realize a proof of concept (PoC) of a system with a mixed-criticality scheduler implemented in the Zephyr RTOS. The system contains non-critical tasks that perform opportunistic processing, represented by CNN applications, and critical tasks with strong time constraints, represented by control-command applications. In more details, the work contains two tasks below.

(1) Provide an implementation of an MCS scheduler in the Zephyr RTOS. We plan to realize two components: MCS-lib and MCS-kernel
- MCS-lib: a set of libraries to facilitate the implementation of MCS tasks.
- MCS-kernel: integration of the MCS scheduler in the Zephyr kernel..

(2) Identify the requirements for deploying one or more CNN applications on an embedded platform with the Zephyr RTOS. Targeted CNN applications and embedded platforms include, but are not limited to, the fol-lowing list:
- UNet for image segmentation
- YOLO for object detection
- Raspberry PI 5 Model B (Quad-core ARM Cortex-A72 processor)

Contact
- Hai Nam TRAN (hai-nam.tran [AT] univ-brest.fr)
- Alan LE BOUDEC (alan.le-boudec [AT] thalesgroup.com)

References
[1]    https://zephyrproject.org/
[2]    Olaf Ronneberger, Philipp Fischer et Thomas Brox, « U-Net: Convolutional Networks for Biomedical Image Segmentation », Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Springer International Publishing,‎ 2015, p. 234–241
[3]    Joseph Redmon, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[4]    J. M. Calandrino, H. Leontyev, A. Block, U. C. Devi and J. H. Anderson, "LITMUS^RT : A Testbed for Empirically Comparing Real-Time Multiprocessor Schedulers," 2006 27th IEEE International Re-al-Time Systems Symposium (RTSS'06), Rio de Janeiro, Brazil, 2006, pp. 111-126, doi: 10.1109/RTSS.2006.27.
[5]    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436-444.
[6]    Alan Le Boudec, Frank Singhoff, Hai Nam Tran, Stéphane Rubini, Sébastien Levieux, and Alexandre Skrzyniarz. "Work-In-Progress: Could Tensorflow applications benefit from a mixed-criticality approach?." In 2023 IEEE Real-Time Systems Symposium (RTSS), pp. 427-430. IEEE, 2023.
 

Master 2 internship. It is preferable for the candidate to have a background or a first-time experience in one of the three domains: (1) Real-time Systems, (2) Embedded programming, (3) Deep learning (convolutional neural network). A strong background in software engineering is well appreciated.