Presentation is available here
I’m showing data and exemples of why dendrites can be a neat mechanisms for biologically inspired innovation to overcome catastrophic forgetting.
Abstract: Dendrites constitute a major unit element of biological neurons and more generally biological neural networks. Their multiple properties have mostly been ignored by the broader neuroscience and machine learning communities due to their lack of conceptual usefulness. So far, the theoretical framework of dendrites has been more or less that of summarizing the synaptic input activity to the soma. However recent experiments in biology, computational modeling and artificial intelligence (AI) have shown that they possess more interesting properties than simply that of linearly summation of weighted synaptic inputs ; driving a renewed interest in their properties and building a strong case for their usefullness. While point neurons have shown tremendous capabilities in deep neural networks, they fail spectaculary in some settings that could be argued are fundamental towards more capable AI, notably in the case of continual learning where there is ongoing memory formation, maintenance and retrieval. While some advances have been made toward the goal of developing algorithms more robust to catastrophic forgetting and able to perform impressive memory-related abilities, we still have a long way to go until we reach the potential of biological networks. There is still much unknown in dendrites’ computational advantages and capabilities but dendritic-enabled unit models are emerging as a biologically-inspired alternative to typical point neuron better able to answer some of those challenging tasks. I will present some of the biological details as well as subtelties of dendrites computation in the brain; then reflect on how they can and are being translated to dit into continual learning research and applications. More precisely reflect on how dendrites have been shown to gate and direct the flow of information in neurons, integrate information over multiple timescales and lead to the emergence of sparse subnetworks.