Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster D in Poster Session D: Wednesday, August 5, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Artifacts as Memory Beyond the Agent Boundary

John D Martin¹, Fraser Mince², Esra'a Saleh³, Amy Pajak⁴; ¹Openmind Research Institute, ²University of Massachusetts at Amherst, ³Cohere Labs Community, ⁴University of Pennsylvania

Presenter: Fraser Mince

Agents routinely use their environment to compensate for cognitive limitations. Here, we begin formalizing this intuition with artificial Reinforcement Learning~(RL) agents. We introduce a mathematical framing for how the environment can functionally serve as an agent's memory, and prove that certain observations, which we call artifacts, reduce the information needed to represent history. We corroborate our theory with computational experiments showing that when agents observe spatial paths, the memory required to learn a performant policy is reduced. Interestingly, this effect arises unintentionally and implicitly through the agent's sensory stream. We also show our findings satisfy qualitative properties previously used to ground accounts of external memory. Moving forward, we anticipate further work on this subject could reveal principled ways to exploit the environment as a substitute for explicit internal memory.

Topic Area: Methods, Tools, Theory & Neural Coding