Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

During natural vision, semantic novelty modulates fixation-related processing in primate cortex

Vinay S Raghavan¹, Jens Madsen^2,3, Maximilian Nentwich⁴, Marcin Leszczynski⁴, Arnaud Falchier⁵, Stephan Bickel³, Brian Russ⁵, Lucas Parra¹; ¹CUNY City College of NY, ²The Feinstein Institutes of Medical Research, ³Northwell Health, ⁴Columbia University College of Physicians and Surgeons, ⁵Nathan Kline Institute

Presenter: Vinay S Raghavan

We sample visual scenes with short gaze fixations separated by saccades. While low-level transsaccadic integration is known, semantic integration across multiple fixations remains unclear. We hypothesized that the brain compares semantic information from one fixation to the next, and therefore postulated a neural signal associated with semantic novelty for each saccade. Novelty was measured using a deep network on foveal vision. Novelty was orthogonal to low- and mid-level visual features and modulated frontal and occipital fixation-related potentials in human EEG during natural viewing of full-length movies (3.4 × 10⁶ saccades). Intracranial recordings in humans (9.0 × 10⁴ saccades) and non-human primates (3.3 × 10⁴ saccades) revealed broadband high-frequency activity modulations in ventromedial visual and frontal brain areas. This modulation was stronger for movies than static images, and frontal modulation preceded occipital modulation, suggesting top-down effects. This ubiquitous modulation of fixation-related activity with novelty suggests that foveal representations are integrated across saccades in primates.

Topic Area: Computational Models of Vision & Visual Cortex