Spiking neural network compatible with ultra-low power hardware for train counting

The FOX research group is part of the CRIStAL laboratory (University of Lille, CNRS), located in Lille, France. We focus on video analysis for human behavior understanding. Specifically, we develop spatio-temporal models of motions for tasks such as abnormal event detection, emotion recognition, and face alignment. We are also involved in IRCICA (CNRS), a research institute promoting multidisciplanary research. At IRCICA, we collaborate with computer scientists and experts in electronics engineering to create new models of neural networks that can be implemented on low-power hardware architectures. Recently, we designed state-of-the-art models for image recognition with single and multi-layer unsupervised spiking neural networks. We were among the first to succesfully apply unsupervised SNNs on modern datasets of computer vision. We also developed our own SNN simulator to support experiments with SNN on computer vision problems. Our work is published in major journals (Pattern Recognition, IEEE Trans. on Affective Computing) and conferences (NeurIPS, WACV, IJCNN) in the field.

The position is located in Lille, France. With over 110 000 students, the metropolitan area of Lille is one France's top education student cities. The European Doctoral College Lille Nord-Pas de Calais is headquartered in Lille Metropole and includes 3,000 PhD Doctorate students supported by university research laboratories. Lille has a convenient location in the European high-speed rail network. It lies on the Eurostar line to London (1:20 hour journey). The French TGV network also puts it only 1 hour from Paris, 35 mn from Brussels, and a short trips to other major centres in France such as Paris, Marseille and Lyon.

An unsupervised spiking-based counting solution has been deployed by the FOX team at Lilliad’s Xperium. The prototype was build using ad-hoc spiking models that do not reflect the requirements of an efficient hardware implementation. In order to reach efficient hardware implementation, the neuron models and the synapse plasticity mechanisms must reflect the existing hardware models. CSAM/ANODE designed ultra-low-power spiking neurons and synapses. In order to take advantage of their energy efficiency, it is important that the simulated model parameters and learning rule fit the hardware model exactly. A first iteration was done last year, where we managed to simulate the expected spiking behavior and pattern learning properties from the hardware specifications on toy examples. This year, based on these initial results, our goal is to develop the neural network model to solve the actual train counting problem using data acquired by our spiking camera.

Spiking neural networks (SNNs), also called third generation neural networks, are a type of neural networks that process information in the form of sparse events, called spikes, instead of traditional computations based on numerical values. This type of processing takes its roots in the way biological neurons (as found in the human brain) work. One compelling property of these spiking networks is the theoretical possibility to build dedicated hardware, called neuromorphic hardware, that implements them and consumes very little energy. Developing such neural networks could lead to dedicated machine learning systems consuming down to a few microwatts. As of today, there exists software models of SNNs that can deal with standard tasks of machine learning (e.g., computer vision, audio processing...), as well as some ultra-low power hardware implementations of some models of spiking neurons; however, the two approaches rely on different neuron and synapse models, preventing the production of actual hardware networks able to learn to solve realistic problems on chip.

The NeuroInspired axis gathers teams from different labs to develop new bio-inspired, energy-efficient models of machine learning. On the one hand, the FOX team (from CRIStAL) develops spiking neural networks models for computer vision applications (object and action recognition, vehicle counting, etc.) [1, 2]. On the other hand, researchers from IEMN design some ultra-low power hardware components that implements specific models of spiking neurons and synapses [3]. Although the software and hardware models share many fundamental properties (integrate-and-fire neuron model, Hebbian learning rules), they also differ in a number of details (for instance, the specifics of the learning rule, or the range of values taken by the parameters of the model) that hampers the direct use of the hardware components to address computer vision tasks.

The objective of this project is to help bridge the gap between software models of SNNs for computer vision, and their actual implementation on ultra-low power, dedicated hardware. The project will address a simple task of computer vision: train counting. A simple Hebbian learning-based software model of SNNs already exists within the FOX team to address this problem in an unsupervised setting. The general goal of this project is to adapt this model to fit exactly the properties of hardware spiking neurons and synapses, as a first step towards producing an actual, ultra-low power hardware implementation of the system. More specifically, it is expected to:

1. Get familiar with some models of spiking neurons and synapses, the existing implementation of the vehicle counting SNN, and our previous results on “hardware-ready” SNN modeling,
2. Design a simulated SNN model to count trains from spiking video data, using the exact hardware models of neurons and synapses,
3. Validate the updated model on some (pre-existing) experimental data.

The long-term goal of the project is to produce an actual prototype of a hardware SNN that learns to count vehicles without supervision and at a minimal energy cost; however, all tasks related to the design of the circuit is beyond the scope of the current project, which will only address the formal model of the SNN addressing this problem.

Experience in one or more of the following is a plus:

 • image processing, computer vision;

 • machine learning;

 • bio-inspired computing;

 • research methodology (literature review, experimentation…).

Candidates should have the following skills:

 • scientific writing;

 • programming (experience in C++ is a plus, but not mandatory).