Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster F in Poster Session F: Thursday, August 6, 1:45 – 3:30 pm, Kimmel Center, Shorin & Rosenthal Rooms

Learning long-range spatial dependencies with spectral convolutional state space models

Drew Linsley¹, Grégoire Dhimoïla¹, Brian Kim¹, Alekh Karkada Ashok¹, Thomas Serre¹; ¹Brown University

Presenter: Grégoire DHIMOÏLA

Visual perception emerges from computations performed over feedforward and recurrent feedback connections between neurons. Yet today's state-of-the-art deep neural networks are almost entirely feedforward. Scaling has favored exclusively feedforward architectures that fit deep learning hardware, even at the cost of biological alignment. Here, we bridge the gap between biology and deep learning at scale with spectral convolutional state space models (SCSSMs), a scalable vision architecture that implements recurrent horizontal connections by lifting convolutional recurrence into the Fourier domain. A single SCSSM layer achieves human-level performance on visual reasoning challenges where transformers fail, and can be scaled up to large-scale vision benchmarks while matching transformer training speeds with fewer parameters and improved performance.

Topic Area: Computational Models of Vision & Visual Cortex