JOM KITA KE POLITEKNIK

Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models (Record no. 2372)

MARC details
042 ## - AUTHENTICATION CODE
Authentication code dc
100 10 - MAIN ENTRY--PERSONAL NAME
Personal name Millidge, Beren
Relator term author
9 (RLIN) 2821
245 00 - TITLE STATEMENT
Title Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models
260 ## - PUBLICATION, DISTRIBUTION, ETC.
Date of publication, distribution, etc. 2022-07.
500 ## - GENERAL NOTE
General note /pmc/articles/PMC7614148/
520 ## - SUMMARY, ETC.
Summary, etc. A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possess close links with self-attention in machine learning. In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection. We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov & Hopfield (2020) to express general associative memory models using neural network dynamics with local computation, and derive a general energy function that is a Lyapunov function of the dynamics. Finally, using our framework, we empirically investigate the capacity of using different similarity functions for these associative memory models, beyond the dot product similarity measure, and demonstrate empirically that Euclidean or Manhattan distance similarity metrics perform substantially better in practice on many tasks, enabling a more robust retrieval and higher memory capacity than existing models.
540 ## - TERMS GOVERNING USE AND REPRODUCTION NOTE
Terms governing use and reproduction
546 ## - LANGUAGE NOTE
Language note en
690 ## - LOCAL SUBJECT ADDED ENTRY--TOPICAL TERM (OCLC, RLIN)
Topical term or geographic name as entry element Article
655 7# - INDEX TERM--GENRE/FORM
Genre/form data or focus term Text
Source of term local
700 10 - ADDED ENTRY--PERSONAL NAME
Personal name Salvatori, Tommaso
Relator term author
9 (RLIN) 2822
700 10 - ADDED ENTRY--PERSONAL NAME
Personal name Song, Yuhang
Relator term author
9 (RLIN) 2823
700 10 - ADDED ENTRY--PERSONAL NAME
Personal name Lukasiewicz, Thomas
Relator term author
9 (RLIN) 2824
700 10 - ADDED ENTRY--PERSONAL NAME
Personal name Bogacz, Rafal
Relator term author
9 (RLIN) 2825
786 0# - DATA SOURCE ENTRY
Note Proc Mach Learn Res
856 41 - ELECTRONIC LOCATION AND ACCESS
Uniform Resource Identifier <a href="/pubmed/36751405">/pubmed/36751405</a>
Public note Connect to this object online.

No items available.