Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster C in Poster Session C: Wednesday, August 5, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Will, Won’t, Want, and Wont: A Reinforcement Learning Model of Self-control

Sahiti Chebolu¹, Tingke Shen², Niklas Buergi¹, Peter Dayan¹; ¹Max Planck Institute for Biological Cybernetics, ²Max Planck Schools

Presenter: Sahiti Chebolu

Achieving long-term goals requires not only planning actions but also persistently executing them in the face of immediate temptations. Failures of self-control have long been studied in otherwise healthy behavior and pathological conditions such as addiction and impulsivity (Ainslie, 2001; Evenden, 1999; Rachlin, 1974; Schelling, 1984). These failures can be mitigated by commitment devices (Duckworth et al., 2018; Fudenberg and Levine, 2006), or by cognitive mechanisms collectively referred to as “willpower” (Ainslie, 2021; Loewenstein, 2000; Robinson et al., 2010). We formalize the self-control problem within a normative reinforcement learning framework in which a prospective planner decides whether to attempt resisting a temptation by accounting for its own uncertainly (in)competent willpower. The planner can improve its efficacy of exerting self-control by training its willpower. It can also enjoy an implicit determination bonus from reducing uncertainty about its degree of self-control. We assess our model’s predictions when repeatedly choosing between a larger-delayed reward, and a smaller-immediate reward. Our simulations show that willpower training and exploration provide a computational account of how agents can overcome their Pavlovian biases (Dayan et al., 2006; Yee and Braver, 2018). We also show that the model captures a range of successes and failures of self-control behavior. Ultimately, explaining self-control lapses using our generative model could aid in the understanding, diagnosis, and treatment of psychiatric conditions.

Topic Area: Decision-Making, Cognitive Control & Event Cognition