Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster C in Poster Session C: Wednesday, August 5, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Trapped in planning: a computational study of episodic memory-guided decision making

Armanjot kaur Randhawa¹, Ming Bo Cai²; ¹Independent researcher, ²University of Miami

Presenter: Ming Bo Cai

Anxiety disorders are characterized by excessive uncontrollable worrying about future events. It has been recently suggested that worry may be viewed as a maladaptive form of planning which normally should support flexible goal-directed behavior in environments that require multi-step decision making. What environmental or cognitive factors may cause an agent to learn to over-plan, or worry excessively in anxiety? We formulate the problem of learning to plan properly as a sequential decision-making problem to balance between planning, working towards the goal and relaxation. Planning, defined here as sampling episodic memory of tasks similar to the current one to estimate action values, can help improve future outcome by providing task-specific estimates of state-action values for future scenarios. However, planning costs time and is unnecessary towards the end of a task. By systematic simulation of reinforcement learning agents, we show that several environmental factors influence the chance that agents learn to over-plan unnecessarily: hard task frequency, task features distinction, magnitude of final outcome, and time limit. The cognitive factors that influence over-planning are: exploitation-exploration tradeoff, memory capacity and sampling efficiency, and weight on values from planning.

Topic Area: Decision-Making, Cognitive Control & Event Cognition