Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

The Latent Dimensions Supporting Dynamic Social Scene Perception

Kathy Garcia¹, Leyla Isik¹; ¹Johns Hopkins University

Presenter: Kathy Garcia

Humans rapidly recognize and understand a rich variety of social scenes, but how this information is organized in the human mind and brain is still largely unknown. We addressed this by collecting 54,000 odd-one-out triplet similarity judgments on an existing dataset of short social interaction videos with behavioral annotations and fMRI data, and training a data-driven embedding model to recover the latent dimensions structuring human social-scene perception. The learned embedding predicted held-out similarity judgments substantially better than human-annotated ratings grounded in established theories of social perception. A compact set of relatively distinct dimensions captured most explainable variance and reflected interpretable social contexts and activities, including conversation, caregiving, cooking, outdoor play, music performance, and infant interaction, while splitting broad categories (e.g. infant scenes) into distinct contexts such as caregiving, play, and crying rather than collapsing them into a single dimension. Interestingly, these same dimensions also predicted neural representational geometry in fMRI data significantly better than predefined behavioral features across visual cortex. Together, these results show an empirically derived, low-dimensional mapping linking behavioral similarity judgments to neural representations for social-scene perception.

Topic Area: Computational Models of Vision & Visual Cortex