Beating Roger Federer: Modeling visual learning and expertise through a bioinspired neural network embedded in an electronic device Goal: PhD grant. When: starting from September 2018 (date may be flexible to some extent) Topic: How do expert tennis players, like Roger Federer for example, predict if a ball will bounce in or out the field to decide if it should be played or not? After thousands of trajectory presentations, best champions have developed extraordinary skills in such a task, but little is known on how the visual system turns selective to spatiotemporal properties of the visual stimulus (e.g., 3D position, velocity and acceleration) and learns how to make an efficient use of it. The goal of the project is to build an embedded system – based on FPGA circuits and ARM processor – an artificial neural network which would replicate – and perhaps beat – the visual and anticipatory performances of these expert players. To achieve this goal successfully, we will develop a bio-inspired neural network, based on some of the key properties of human vision: the Smart NeuroCam (GST company) will be used to reproduce the retina functioning. It triggers its message under the form of spikes, in an asynchronous way (without any concept of frame per second), responding to spatial or temporal changes in the pattern of illumination. Several kinds of pre-processing filters can be implemented in VHDL language directly in the FPGA circuits, and the output is then sent to a neural network. The artificial network will learn to use this message, applying a simple learning rule, the Spike-Timing-Dependent Plasticity (STDP). This rule allows each neuron to become selective to a particular property of the stimulus, completely autonomously and with no supervision. Several layers will be built to allow perceiving more and more complex properties of the visual scene. Once the network will be established, its performances will be assessed in different conditions of learning and compared to those of the best tennis players. The project is funded by a French National Research Agency (ANR), involving two sites and several researchers: − Robin Baurès, Benoit Cottereau, Timothée Masquelier and Simon Thorpe, CerCo, Toulouse. − Michel Paindavoine, GST, Dijon. Where: The candidate will be based at CerCo, Toulouse (France), and will make the interface with the two sites, with regular trips. The computational neuroscience part will be done at Toulouse, and electronic part at Dijon. Tasks: − Matlab (or Python) based simulations of numerical filters. These filters will be applied to the image processing from which spikes are generated and then sent to feed the neural network and STDP learning mechanism. − VHDL coding to implement these numerical filters into the FPGA circuits of the cameras − C/C++ coding of the neural network and STDP mechanism that should work on an embedded ARM processor system − Experimental tests that will allow evaluating the performance of the whole system, from spikes generation to visual properties learning of the embedded system, to predict tennis ball’s trajectories Required skills: − Strong knowledge on electronic, and openness to computational neurosciences − Knowledge in signal-image processing, and artificial neural network − Interest for multidisciplinary research − Ability to turn smoothly autonomous, once the road has been set − Ability to be at the interface of two scientific fields and two working areas − Programming with Matlab and/or Python for simulating the neural network − Programming in VHDL language for FPGA circuits − Programming in C/C++ language for porting the algorithms on ARM microprocessors − French is not a requirement if fluent in English, but willingness to learn would be beneficial Citizenship Eligibility: For security / defense reasons, an evaluation will be made of the candidates before any possible acceptation. Relevant publications for the project: − Masquelier, T., Guyonneau, R. & Thorpe S.J. (2009). Competitive STDP-Based Spike Pattern Learning. Neural Comput, 21(5),1259-1276. − Masquelier, T. & Thorpe, S.J. (2007). Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Comput Biol, 3(2):e31. − Cottereau, B.R., McKee, S.P. & Norcia, A.M. (2014). Dynamics and cortical distribution of neural responses to 2D and 3D motion in human. Journal of Neurophysiology 111(3), 533-543. − SmartNeuroCam de GST : https://gsensing.eu/fr/category/sections/products Contact: Robin Baurès, PhD Associate Professor CerCo, Université Toulouse 3, CNRS CHU Purpan, Pavillon Baudot 31059 Toulouse Cedex 9 – France Office phone: 0033 (0)5 62 74 62 15 Email : robin.baures@cnrs.fr Pr Michel Paindavoine GlobalSensing Technologies 14, rue Pierre de Coubertin 21000 Dijon email : michel.paindavoine@gsensing.eu -- Timothée Masquelier - timothee.masquelier@cnrs.fr - http://cerco.ups-tlse.fr/~masquelier/