Publication
Learning, Inference, and Replay of Hidden State Sequences in Recurrent Spiking Neural Networks
Related concepts (32)
Recurrent neural network
A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.
Types of artificial neural networks
There are many types of artificial neural networks (ANN). Artificial neural networks are computational models inspired by biological neural networks, and are used to approximate functions that are generally unknown. Particularly, they are inspired by the behaviour of neurons and the electrical signals they convey between input (such as from the eyes or nerve endings in the hand), processing, and output from the brain (such as reacting to light, touch, or heat). The way neurons semantically communicate is an area of ongoing research.
Reservoir computing
Reservoir computing is a framework for computation derived from recurrent neural network theory that maps input signals into higher dimensional computational spaces through the dynamics of a fixed, non-linear system called a reservoir. After the input signal is fed into the reservoir, which is treated as a "black box," a simple readout mechanism is trained to read the state of the reservoir and map it to the desired output. The first key benefit of this framework is that training is performed only at the readout stage, as the reservoir dynamics are fixed.
Spiking neural network
Artificial neural network Spiking neural networks (SNNs) are artificial neural networks that more closely mimic natural neural networks. In addition to neuronal and synaptic state, SNNs incorporate the concept of time into their operating model. The idea is that neurons in the SNN do not transmit information at each propagation cycle (as it happens with typical multi-layer perceptron networks), but rather transmit information only when a membrane potential—an intrinsic quality of the neuron related to its membrane electrical charge—reaches a specific value, called the threshold.
Autonomic nervous system
The autonomic nervous system (ANS), formerly referred to as the vegetative nervous system, is a division of the nervous system that supplies internal organs, smooth muscle and glands. The autonomic nervous system is a control system that acts largely unconsciously and regulates bodily functions, such as the heart rate, its force of contraction, digestion, respiratory rate, pupillary response, urination, and sexual arousal. This system is the primary mechanism in control of the fight-or-flight response.
Parasympathetic nervous system
The parasympathetic nervous system (PSNS) is one of the three divisions of the autonomic nervous system, the others being the sympathetic nervous system and the enteric nervous system. The enteric nervous system is sometimes considered part of the autonomic nervous system, and sometimes considered an independent system. The autonomic nervous system is responsible for regulating the body's unconscious actions.
Somatic nervous system
The somatic nervous system (SNS), or voluntary nervous system is the part of the peripheral nervous system associated with the voluntary control of body movements via skeletal muscles. The somatic nervous system consists of nerves carrying afferent nerve fibers, which relay sensation from the body to the central nervous system (CNS), and nerves carrying efferent nerve fibers, which relay motor commands from the CNS to stimulate muscle contraction. The a- of afferent and the e- of efferent correspond to the prefixes ad- (to, toward) and ex- (out of).
Peripheral nervous system
The peripheral nervous system (PNS) is one of two components that make up the nervous system of bilateral animals, with the other part being the central nervous system (CNS). The PNS consists of nerves and ganglia, which lie outside the brain and the spinal cord. The main function of the PNS is to connect the CNS to the limbs and organs, essentially serving as a relay between the brain and spinal cord and the rest of the body. Unlike the CNS, the PNS is not protected by the vertebral column and skull, or by the blood–brain barrier, which leaves it exposed to toxins.
Residual neural network
A Residual Neural Network (a.k.a. Residual Network, ResNet) is a deep learning model in which the weight layers learn residual functions with reference to the layer inputs. A Residual Network is a network with skip connections that perform identity mappings, merged with the layer outputs by addition. It behaves like a Highway Network whose gates are opened through strongly positive bias weights. This enables deep learning models with tens or hundreds of layers to train easily and approach better accuracy when going deeper.
Vanishing gradient problem
In machine learning, the vanishing gradient problem is encountered when training artificial neural networks with gradient-based learning methods and backpropagation. In such methods, during each iteration of training each of the neural networks weights receives an update proportional to the partial derivative of the error function with respect to the current weight. The problem is that in some cases, the gradient will be vanishingly small, effectively preventing the weight from changing its value.