Hopfield network

A Hopfield network (or Amari-Hopfield network, Ising model of a neural network or Ising–Lenz–Little model) is a form of recurrent artificial neural network and a type of spin glass system popularised by John Hopfield in 1982 as described by Shun'ichi Amari in 1972 and by Little in 1974 based on Ernst Ising's work with Wilhelm Lenz on the Ising model. Hopfield networks serve as content-addressable ("associative") memory systems with binary threshold nodes, or with continuous variables. Hopfield networks also provide a model for understanding human memory. The Ising model of a recurrent neural network as a learning memory model was first proposed by Shun'ichi Amari in 1972 and then by William A. Little in 1974, who was acknowledged by Hopfield in his 1982 paper. Networks with continuous dynamics were developed by Hopfield in his 1984 paper. A major advance in memory storage capacity was developed by Krotov and Hopfield in 2016 through a change in network dynamics and energy function. This idea was further extended by Demircigil and collaborators in 2017. The continuous dynamics of large memory capacity models was developed in a series of papers between 2016 and 2020. Large memory storage capacity Hopfield Networks are now called Dense Associative Memories or modern Hopfield networks. The units in Hopfield nets are binary threshold units, i.e. the units only take on two different values for their states, and the value is determined by whether or not the unit's input exceeds its threshold . Discrete Hopfield nets describe relationships between binary (firing or not-firing) neurons . At a certain time, the state of the neural net is described by a vector , which records which neurons are firing in a binary word of bits. The interactions between neurons have units that usually take on values of 1 or −1, and this convention will be used throughout this article. However, other literature might use units that take values of 0 and 1. These interactions are "learned" via Hebb's law of association, such that, for a certain state and distinct nodes but .