Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Contrastive Self-Supervised Learning in Higher Visual Cortex

Daniel D. Kato¹, Sreyas P. Adiraju¹, Ashley C. Green¹, Elias Issa¹; ¹Columbia University

Presenter: Daniel D. Kato

Models of self-supervised learning (SSL) have proven remarkably successful in machine vision, suggesting themselves as a model of primate vision. However, studies of learning in the brain have thus far yielded few direct observations of a signature hallmark of SSL: bidirectional learning in the absence of external supervision. In this study, we report that neural responses in inferior temporal (IT) cortex of common marmoset monkeys (_Callithrix jacchus_) indeed undergo opposite learning effects over monthslong exposure to naturalistic scenes via two distinct modes of self-supervised viewing: 1) through static eye fixation, and 2) across saccadic eye movements. We find that image pairs repeatedly shown in quick succession through stable fixation become less neurally discriminable from each other compared to novel controls, while images repeatedly viewed across saccades became more discriminable. Thus, a biological implementation of the internal SSL signal relies on the distinction between periods of eye movement versus stationarity.

Topic Area: Computational Models of Vision & Visual Cortex