Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster F in Poster Session F: Thursday, August 6, 1:45 – 3:30 pm, Kimmel Center, Shorin & Rosenthal Rooms

Evaluating spatial perception in humans and machines

Nathan Kong¹, Václav Knapp², Russell A. Epstein¹, tyler bonnen²; ¹University of Pennsylvania, ²University of California, Berkeley

Presenter: Nathan Kong

Humans perceive spatial structure in the environment. This ability supports a range of downstream behaviours—from navigation to memory retrieval—and is thought to rely on a network of 'scene-selective' cortical structures. Feedforward deep learning models are often thought to provide a suitable approximation of this perceptual ability. This human-model correspondence, however, has largely been evaluated in classification settings. Here we develop a novel behavioural assay which reveals a substantial gap between these vision models and human spatial perception. We procedurally generate a large set of scenes and format them into 'oddity' tasks: participants are presented with two viewpoints from one tabletop (A), along with an image from a different tabletop (B), and must identify the odd-one-out (B). Through a series of experiments, we find that humans substantially outperform state-of-the-art vision models on this benchmark, and that human reaction times scale linearly with task difficulty. These data highlight the temporal dynamics of spatial perception, challenging common assumptions about the feedforward underpinnings of this foundational human ability.

Topic Area: Computational Models of Vision & Visual Cortex