Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

A Benchmark for Physical Object Relationship Perception in Humans and Machines

Georgina Woo¹, RT Pramod¹, Nancy Kanwisher¹; ¹Massachusetts Institute of Technology

Presenter: Georgina Woo

We introduce a benchmark dataset of synthetic 3D-rendered images spanning five physical object relationships (Attach, Contain, Cover, Separate, Support), each instantiated across varied objects, materials, viewpoints, and backgrounds, and measure human and model performance on a triplet matching task requiring cross-object generalization. Humans perform near-ceiling (97.2%) while all models fall significantly short, with the best model (V-JEPA2) reaching only 83.5%. Moreover, human and model error patterns across relationship pairs are uncorrelated, suggesting fundamental differences in how the two systems solve the task and pointing to important limitations in current visual representations of physical relationships.

Topic Area: Computational Models of Vision & Visual Cortex