Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster D in Poster Session D: Wednesday, August 5, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Continuous Temporal Difference Learning as a Unifying Theory of Dopamine Function Across Timescales

Sankalp Garud¹, Laurel Morris^1,2; ¹University of Oxford, ²Icahn School of Medicine at Mount Sinai

Presenter: Sankalp Garud

Dopamine neurons are thought to signal reward prediction errors phasically and the opportunity cost of time tonically, while also displaying puzzling ramping activity during goal approach, and coupling with movement or velocity. These are often treated as distinct modes of dopamine function, each requiring its own computational explanation. Here we show that all can be unified by considering temporal difference learning within the context of continuous time, combined with the assumption that the brain computes value changes through a fast model-based process while maintaining a slower model-free cache. Together, the inclusion of these two ingredients explains phasic responses, tonic modulation between reward contexts, navigation ramps, speed scaling, and the fading of ramps with learning--without invoking separate mechanisms. We confirm these predictions across two independent datasets spanning freely-moving and head-fixed paradigms. Continuous temporal difference learning may thus provide a unified theory of dopamine function.

Topic Area: Methods, Tools, Theory & Neural Coding