From Human Cognition to Continual Learning: Neural Selectivity, Memory Consolidation and Dreaming

Continual learning addresses the problem of incrementally acquiring knowledge from a sequence of tasks or a non-stationary data stream, without retraining from scratch. Despite a wide range of proposed strategies, neural networks trained sequentially often suffer from catastrophic forgetting, i.e., the tendency to lose performance on previously learned tasks after learning new ones. On the contrary, humans have a remarkable ability to learn continuously, retaining past experiences while quickly adapting to new tasks and problems. This gap between artificial and biological learning can be attributed to the inherent structure and optimization approaches of neural networks, which differ significantly from the way humans learn and build neural connectivity over a lifetime. Motivated by this discrepancy, this dissertation investigates biologically inspired mechanisms for continual learning, leveraging neuroscientific insights as design principles to improve retention while facilitating future adaptation. Architectural inductive biases inspired by neural selectivity and pattern separation are investigated to promote more compartmentalized representations that reduce interference between tasks. Drawing inspiration from synaptic consolidation, strategies that selectively constrain updates to parts of the network are explored to support rapid adaptation while preserving previously acquired knowledge. This principle is then developed within a wake-sleep cycle grounded in Complementary Learning Systems (CLS) theory, where online learning is complemented by offline replay and multi-timescale memory organization. Within this view, replay and dreaming are conceived as functional components that support the restructuring of knowledge, which in turn lays the groundwork for future learning. This perspective is further developed through an approach in which dreams are generated directly from learned representations, enabling the autonomous construction of more adaptive representations. Across standard benchmarks, the experimental results demonstrate that integrating neuroscience-inspired concepts into continual learning methods improves performance, strengthening both the preservation of past knowledge and the ability to learn from new tasks.