Sleep is a pretty good thing, we’ve already discussed that. In humans, scientific evidence suggests there are a huge range of benefits, including something as important as keeping the brain healthy by allowing neurons to remove unnecessary synaptic connections that we make during the day.
This process, called synaptic homeostasis, prevents the brain from being invaded by useless memories. It may help improve our cognitive performance, while dreams allow us to process our memories.
As a result, something similar may be happening when artificial neural networks can sleep and dream. That’s right, and it works in much the same way as how it is believed to occur in humans.
Of course, artificial neural networks, a type of artificial intelligence based on biological neural networks, do not automatically and instinctively sleep or dream. That’s why a team of mathematicians from Italy programmed a type of artificial neural network called a Hopfield network so that it could sleep.
“Inspired by the sleep mechanisms of mammalian brains, we propose an extension of this model that shows the standard online (wakefulness) learning mechanism (which allows the storage of external information in terms of patterns) and an offline one ( sleep), a mechanism of unlearning and consolidation, “write the authors in their article published in the journal Neural Networks .
In other words, while the artificial neural network is “awake,” it is learning and storing patterns; but its storage capacity is limited. So the team discovered a way to mathematically implement human sleep patterns: rapid eye movement sleep and slow wave sleep; Regarding the former, it is believed that it eliminates unnecessary memories, and the latter is believed to consolidate the important ones.
So this is what the “resting” state of the artificial neural network also does: in that resting state, it reviews and unlearns unnecessary information, and then consolidates what remains, the most remarkable.
The result was incredible. Without sleep, the maximum capacity was α = 0.14, where α represents the number of bits stored per synapse. When a rest cycle was incorporated, the artificial neural network reached the theoretical limit for networks of this type – α = 1.
Extensive testing with simulations validated the result, showing that allowing a neural network to nap every now and then (using the correct algorithm) could improve performance.
“We believe that in the process of cognition, although certainly learning and recovery continue to play a fundamental role, sleep is also mandatory for artificial intelligence, as it is for biological intelligence”, comments mathematician Adriano Barra, from the University of Salento and work leader.
What will your dreams be like? Will they dream of electric sheep?
Referencia:Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones. Neural Networks 2019 Alberto Fachechia, Elena Agliari, Adriano Barra et al. DOI: https://doi.org/10.1016/j.neunet.2019.01.006
