Why some Artificial Brains made of spiking neural networks Need to Sleep

Marees

Gawd
Joined
Sep 28, 2018
Messages
991
"Our results make the surprising prediction that slow-wave sleep may be essential for any spiking neural network, or indeed any organism with a nervous system, to be able to learn from its environment."

Do they dream of electronic sheep, when they sleep ?

https://www.insidescience.org/news/why-artificial-brains-need-sleep

Scientists found that although Spiking Neural Networks could learn to identify the data it was trained to look for, when such training went uninterrupted long enough, its neurons began to continuously fire no matter what signals they received.

Watkins recalled that "almost in desperation," they tried having the simulation essentially undergo deep sleep. They exposed it to cycles of oscillating noise, roughly corresponding to the slow brain waves seen in deep sleep, which restored the simulation to stability. The researchers suggest this simulation of slow-wave sleep may help "prevent neurons from hallucinating the features they're looking for in random noise," Watkins said.

SPIKING Neural Network:
In most artificial neural networks, a neuron's output is a number that alters continuously as the input it is fed changes. This is roughly analogous to the number of signals a biological neuron might fire over a span of time.

In contrast, in a spiking neural network, a neuron "spikes," or generates an output signal, only after it receives a certain amount of input signals over a given time, more closely mimicking how real biological neurons behave.

Since spiking neural networks only rarely fire spikes, they shuffle around much less data than typical artificial neural networks and in principle require much less power and communication bandwidth.
 

erek

Supreme [H]ardness
Joined
Dec 19, 2005
Messages
7,743
Do they dream of electronic sheep, when they sleep ?

https://www.insidescience.org/news/why-artificial-brains-need-sleep

Scientists found that although Spiking Neural Networks could learn to identify the data it was trained to look for, when such training went uninterrupted long enough, its neurons began to continuously fire no matter what signals they received.

Watkins recalled that "almost in desperation," they tried having the simulation essentially undergo deep sleep. They exposed it to cycles of oscillating noise, roughly corresponding to the slow brain waves seen in deep sleep, which restored the simulation to stability. The researchers suggest this simulation of slow-wave sleep may help "prevent neurons from hallucinating the features they're looking for in random noise," Watkins said.

SPIKING Neural Network:
In most artificial neural networks, a neuron's output is a number that alters continuously as the input it is fed changes. This is roughly analogous to the number of signals a biological neuron might fire over a span of time.

In contrast, in a spiking neural network, a neuron "spikes," or generates an output signal, only after it receives a certain amount of input signals over a given time, more closely mimicking how real biological neurons behave.

Since spiking neural networks only rarely fire spikes, they shuffle around much less data than typical artificial neural networks and in principle require much less power and communication bandwidth.

Further coverage on this topic bump:

Artificial Brains Need Sleep Too – Desperate AI Researchers Discover Way to Stabilize Neuromorphic Processors
 
Top