Michael Buice

Allen Institute

Abstract:  Learning in biological systems takes place in contexts and with dynamics not often accounted for by simple models. I will describe the learning dynamics of two model systems that incorporate either architectural or dynamic constraints from biological observations. In the first case, inspired by the observed mesoscopic structure of the mouse brain as revealed by the Allen Mouse Brain Connectivity Atlas, as well as multiple examples of parallel pathways in mammalian brains, I present a mathematical analysis of learning dynamics in networks that have parallel computational pathways driven by the same cost function. We use the approximation of deep linear networks with large hidden layer sizes to show that, as the depth of the parallel pathways increases, different features of the training set (defined by the singular values of the input-output correlation) will typically concentrate in one of the pathways. This result is derived analytically and demonstrated with numerical simulation with both linear and non-linear networks. Thus, rather than sharing stimulus and task features across multiple pathways, parallel network architectures learn to produce sharply diversified representations with specialized and specific pathways, a mechanism which may hold important consequences for codes in both biological and artificial systems. In the second case, I discuss learning dynamics in a generalization of Hebbian rules and show that these rules allow a neuron to learn tensor decompositions of higher-order input correlations. Unlike the case of the Oja rule and PCA, the resulting learned representation is not unique but selects amongst the tensor eigenvectors according to initial conditions.

About VVTNS : Created as the World Wide Neuroscience Seminar (WWTNS) in November 2020 and renamed in homage to Carl van Vreeswijk in Memoriam (April 20, 2022), its aim is to be a platform to exchange ideas among theoreticians. Speakers have the occasion to talk about theoretical aspects of their work which cannot be discussed in a setting where the majority of the audience consists of experimentalists. The seminars, held on Wednesdays at 11 am ET,  are 45-50 min long followed by a discussion. The talks are recorded with authorization of the speaker and are available to everybody on our YouTube channel.