Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Model-derived Minimal Image Pairs Predict Differential Responses in Human Scene-Selective Cortex

Junxia Wang¹, Mainak Deb¹, Haider Al-Tahan¹, Diego Garcia Cerdas², Iris Groen³, Apurva Ratan Murty¹; ¹Georgia Institute of Technology, ²Goethe University Frankfurt, ³University of Amsterdam

Presenter: Junxia Wang

Looking at scenes engages multiple scene-selective regions in the brain, including the parahippocampal place area (PPA), retrosplenial cortex (RSC), and occipital place area (OPA). However, what distinguishes their computations remains less well understood. Prior work has largely relied on cognitive dissociations, which identify differences in when regions may respond but not what visual features drive functional differences. Here, we employ a novel approach: we use computational models to generate images predicted to differentially modulate activity between scene-selective regions. We built encoding models based on CLIP embeddings for the PPA, OPA, and RSC and found that they predicted responses across subjects, while also showing substantial region specificity. We then used the directions in CLIP embedding space, together with a diffusion-based generative method, to synthesize visually similar image pairs (i.e., minimal pairs) predicted to enhance responses in one region while suppressing it in another. We found that the predicted differential response modulation generalizes across subjects, particularly for comparisons involving RSC, while dissociations between OPA and PPA were weaker. The resulting images suggest that the relevant visual feature differences depend on the pair of regions being compared. Our framework demonstrates how models can be taken beyond their predictivity to directly construct new images or hypotheses that may help reveal what different brain regions compute.

Topic Area: Computational Models of Vision & Visual Cortex