Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster D in Poster Session D: Wednesday, August 5, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Using deep reinforcement learning to reveal neural representations of exploration

Tamir Scherf¹, Kristoffer C. Aberg¹, Michal Ramot¹, Rony Paz¹; ¹Weizmann Institute of Science

Presenter: Tamir Scherf

A key aspect of learning and decision-making is balancing the exploitation of known options with the exploration of potentially better ones, a dilemma known as the exploration-exploitation trade-off. Reinforcement learning (RL) theory explains exploration behavior but not its neural representations. To address this, we leveraged deep reinforcement learning (Deep-RL). We used an LSTM-based actor-critic model and fMRI data from participants performing a restless three-armed bandit task. The model revealed a representational structure in which action representations were more similar following low outcome values and became increasingly separated following high outcome values, consistent with a transition between exploratory and exploitative states. This structure was expressed in value-sensitive regions, including dACC, anterior insula, and vmPFC, beyond simpler control models. Critically, value-dependent changes in across-action neural distance in vmPFC were linked to changes in exploratory behavior, suggesting a mechanism similar to that of the model.

Topic Area: Decision-Making, Cognitive Control & Event Cognition