Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Origins of Category Selectivity: Natural Image Statistics Are Necessary but Not Sufficient

Bowen Zheng¹, Meenakshi Khosla², Nancy Kanwisher¹; ¹Massachusetts Institute of Technology, ²University of California, San Diego

Presenter: Bowen Zheng

How do category-selective responses arise in primate visual cortex? We used artificial neural networks to ask what conditions are necessary for category selectivity to emerge. We define three increasingly stringent measures of selectivity, the strictest of which isolates category information beyond low-level visual features. We compare untrained, self-supervised, and supervised ResNet-18 networks, with the latter two trained on Ecoset. By less stringent measures, all trained networks show more selectivity above untrained networks. By our strictest measure, primate ventral visual cortex contains clear category selectivity and supervised networks approach primate levels, but self-supervised networks show weaker selectivity, and random networks show none. To determine what drives the residual selectivity in self-supervised networks, we manipulated the training data in three ways. Removing a category from the training set eliminates residual selectivity, showing that experience with that category is necessary. Training on isolated objects stripped of their scene context significantly reduces residual selectivity, suggesting that scene structure contributes to the category signal. Training on grayscale images does not reduce selectivity. These results show that natural image statistics give rise to partial category selectivity without supervision, but closing the gap to primate levels requires a category-level learning signal.

Topic Area: Computational Models of Vision & Visual Cortex