Scientific context:

Intrinsically motivated goal exploration algorithms enable machines to discover repertoires of action policies that produce a diversity of effects in complex environments. In robotics, these exploration algorithms have been shown to allow real world robots to acquire skills such as tool use in high-dimensional continuous state and action spaces (e.g. Baranes and Oudeyer, 2013; Forestier et al., 2017). An example is the Poppy/Explauto experiment with a humanoid robot learning to use tools in a few hours, which obtained the 2^nd position in the demonstration competition at NIPS 2016 (See   https://www.youtube.com/watch?v=NOLAwD4ZTW0 , Forestier et al., 2016). In other domains such as chemistry and physics, they open the possibility to automate the discovery of novel chemical or physical structures produced by complex dynamical systems (e.g. Grizou, Points et al., 2017)

However, they have so far assumed that self-generated goals are sampled in a specifically engineered feature space, limiting their autonomy. Recent work has shown how unsupervised deep learning approaches could be used to learn goal space representations (e.g. Péré et al., 2018), but they have focused on goals represented as static target configurations of the environment in robotics sensorimotor spaces.

In this project, we will study how these new families of machine learning algorithms can be extended and used for automated discovery of behaviours of dynamical systems in physics/chemistry, through exploration of their parameter space, and when these behaviours are perceived through raw sensors (e.g. pixels of images).

Web site of Flowers Lab: https://flowers.inria.fr

Post-doctoral researcher work description 

The work will begin by a familiarization with algorithms for intrinsically motivated exploration with unsupervised learning of goal spaces developed in the team. Then, a few interesting numerical/simulation models of physical/chemical phenomena, associated to rich behavioral patterns (such as Turing-like reaction-diffusion models or oil droplet models), will be selected as a basis for developing and experimenting updates of existing exploration and learning algorithms. These algorithmic updates will focus in particular on the use of recurrent neural networks to learn goal spaces that can encode dynamical phenomena. Several algorithmic implementations will be compared, e.g. using several techniques for encoding times series as goal spaces, and compared using various information theoretic measures of the diversity of discovered behaviours in the target dynamical systems. It will be also possible to test these algorithmic updates in more traditional (simulated) sensorimotor learning problems, using standard benchmarks of the literature (e.g. Open Gym environments or Minecraft/Malmo).

Experiments will possibly extend the DeepExplauto open-source library developed in the team, as well as standard Deep Learning libraries for the algorithms.

Required knowledge and background:

Candidates should have a strong expertise in at least one of these areas:
- Experience with Deep Learning algorithms (theory and practical implementations)
- Stochastic optimization, black-box optimization
- Intrinsically motivated exploration algorithms

Knowledge and competence in physics/chemical modelling would be welcome.

Other requirements:
- strong skills in mathematics, statistical inference, machine learning
- Advanced programming skills in script languages like python
- Motivation to work in an interdisciplinary project

References :

Baranes, A., & Oudeyer, P. Y. (2013). Active learning of inverse models with intrinsically motivated goal exploration in robots. Robotics and Autonomous Systems, 61(1), 49-73.

Forestier, S., Mollard, Y., Caselli, D., & Oudeyer, P. Y. (2016). Autonomous exploration, active learning and human guidance with open-source Poppy humanoid robot platform and Explauto library. In The Thirtieth Annual Conference on Neural Information Processing Systems (NIPS 2016).

Grizou, J., Point, L. and the Cronin lab (2017) Tutorial on “Developmental robotics in a chemistry lab », https://croningp.github.io/tutorial_icdl_epirob_2017/.

Péré, A., Forestier, S., Oudeyer, P-Y. (2018) Unsupervised learning of goal spaces for intrinsically motivated goal exploration, in proceedings of ICLR 2018.

Points, L. J., Taylor, J. W., Grizou, J., Donkers, K., & Cronin, L. (2018). Artificial intelligence exploration of unstable protocells leads to predictable properties and discovery of collective behavior. Proceedings of the National Academy of Sciences, 201711089.

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.

Keywords: deep learning; intrinsically motivated exploration; unsupervised learning; automated discovery; complex systems; dynamical systems

Pierre-Yves Oudeyer