Spatial Memory in RL Agents

Description

This lecture explores the use of spatial and pattern memory in reinforcement learning agents through maze navigation tasks. The approach involves designing mazes of increasing complexity to test spatial orientation and path choosing abilities. The RL agent is based on a Dueling Network architecture with Double Q-learning for state updates. Results show that increasing visual landmarks improve agent performance in spatial orientation, while path choosing remains inconsistent. Limitations include the lack of environment map building and the need for better reward functions. Possible extensions involve refining starting points and orientations.

Official source

https://mediaspace.epfl.ch/media/0_649cod69

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (7)

Related concepts (79)

Spatial memory

In cognitive psychology and neuroscience, spatial memory is a form of memory responsible for the recording and recovery of information needed to plan a course to a location and to recall the location of an object or the occurrence of an event. Spatial memory is necessary for orientation in space. Spatial memory can also be divided into egocentric and allocentric spatial memory. A person's spatial memory is required to navigate around a familiar city. A rat's spatial memory is needed to learn the location of food at the end of a maze.

Memory

Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered, it would be impossible for language, relationships, or personal identity to develop. Memory loss is usually described as forgetfulness or amnesia. Memory is often understood as an informational processing system with explicit and implicit functioning that is made up of a sensory processor, short-term (or working) memory, and long-term memory.

Long-term memory

Long-term memory (LTM) is the stage of the Atkinson–Shiffrin memory model in which informative knowledge is held indefinitely. It is defined in contrast to short-term and working memory, which persist for only about 18 to 30 seconds. LTM is commonly labelled as "explicit memory" (declarative), as well as "episodic memory," "semantic memory," "autobiographical memory," and "implicit memory" (procedural memory). The idea of separate memories for short- and long-term storage originated in the 19th century.

Visual memory

Visual memory describes the relationship between perceptual processing and the encoding, storage and retrieval of the resulting neural representations. Visual memory occurs over a broad time range spanning from eye movements to years in order to visually navigate to a previously visited location. Visual memory is a form of memory which preserves some characteristics of our senses pertaining to visual experience. We are able to place in memory visual information which resembles objects, places, animals or people in a mental image.

False memory syndrome

In psychology, false memory syndrome (FMS) was a controversial proposed condition in which a person's identity and relationships are affected by what are believed to be false memories of psychological trauma, recollections which are strongly believed but factually contested by the accused. Peter J. Freyd originated the term partly to explain what he said was a false accusation of sexual abuse made against him by his daughter Jennifer Freyd and his False Memory Syndrome Foundation (FMSF) subsequently popularized the concept.