Poster Presentation

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Learning Optimally Fast: a Normative Theory Balancing Effort and Performance

Rodrigo Carrasco-Davis¹, Valentina Njaradi², Peter E. Latham², Andrew M Saxe²; ¹Princeton University, ²University College London

Presenter: Rodrigo Carrasco-Davis

Learning how to learn efficiently is a fundamental problem for biological agents and an increasingly important one for modern machine learning systems. To learn efficiently, an agent must determine how fast to learn, balancing the benefits of rapid improvement against the costs of effort, instability, or resource depletion. We study a normative objective for optimal learning rate control: maximizing cumulative reward throughout learning while accounting for the costs of effort. Using control theory, we derive a closed-form expression for the optimal learning rate that depends only on quantities available to the agent, specifically the current and expected final performance. Remarkably, under mild continuity assumptions alone, this solution holds for arbitrary tasks, network architectures, cost functions, and learning dynamics. We verify that it matches numerically optimized schedules in deep networks. Because the optimal learning rate depends on the agent’s expectation of its future performance, our framework also predicts how over- or under-confidence shapes engagement and effort allocation. These predictions align with findings from self-regulated learning literature, where high performance expectations lead to greater motivation, engagement, and persistence, while low expectations reduce effort. Finally, we introduce a simple episodic-memory mechanism that allows agents to estimate future performance on unseen tasks by recalling similar past trajectories, providing near-optimal inputs to the optimal learning rate expression. Furthermore, performance prediction and prediction errors could be encoded by dopaminergic and serotonergic signals, as both neuromodulators are involved in long-term prediction errors and motivational states related to beliefs in future performance. Together, these results provide a normative and biologically plausible account of learning speed control, linking self-regulated learning, effort allocation, and episodic memory estimation within a unified and tractable mathematical framework.

Topic Area: Methods, Tools, Theory & Neural Coding