Keynotes | K&Ts | GACs | Talks | Posters | Search
Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms
Performance Matters More Than You Think: A Saturation-Effect Resolution to the Predictivity Debate
Khaled Jedoui Al-Karkari1, Josh Wilson1, Daniel LK Yamins1; 1Stanford University
Presenter: Khaled Jedoui Al-Karkari
The hypothesis that neural networks optimized for visual tasks develop cortical-like representations (Yamins et al., 2014) has shaped a decade of computational neuroscience. As models have grown in scale and diversity, the question of whether better task performance actually produces stronger neural alignment has drawn increasing attention. Yet the evidence is contradictory: Schrimpf et al. (2018) found that ImageNet accuracy correlates with neural predictivity but the relationship weakens among top-performing models, and Conwell et al. (2024) concluded that task objective contributes minimally. Here we show that this discrepancy reflects saturation effects that vary across the cortical hierarchy. Evaluating 86 models from 10 architecture families on human 7T fMRI (NSD, 6 hierarchy levels) and macaque electrophysiology (TVSD, 3 regions), we find that the correlation between task accuracy and brain-predictivity increases monotonically: r = 0.53 at V1, 0.69 at V4, and 0.78 at IT in macaques, and r = 0.65 to 0.85 across six levels in humans. This ascending gradient is conserved across species and generalizes to action recognition, confirming that task competence is a primary determinant of neural alignment, obscured when collapsing across cortical levels.
Topic Area: Computational Models of Vision & Visual Cortex