Keynotes | K&Ts | GACs | Talks | Posters | Search

Contributed Talk Session: Tuesday, August 4, 2:00 – 3:00 pm, Skirball Theater
Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

What Makes a Category Memorable? Human Conceptual Diversity Outpredicts Perceptual Similarity and Deep Neural Networks

Dyllan Simpson¹, Mariagrazia De gioia², Bria Lorelle Long¹, Timothy Brady¹; ¹University of California, San Diego, ²University of Bologna

Presenter: Dyllan Simpson

Conceptual knowledge structures human memory, with better memory performance for more conceptual diverse information even when it is perceptually similar. To what extent do modern deep neural networks capture the conceptual structure relevant for memory performance? We investigated how human judgments of within-category diversity relate to memorability and to the representational structure of deep neural networks. Participants sorted exemplar images from 498 object categories (from the THINGS database) based on either conceptual similarity (function, purpose, typical context) or perceptual similarity (shape, color, texture). We measured category-level memorability using a recognition memory task with within-category foils. We then tested how memory performance related to human diversity scores and to diversity computed from CLIP, DINOv3, and VGG16 embeddings across multiple layers. Human conceptual and perceptual diversity were correlated (r = .46) but dissociable, with some categories showing high conceptual but low perceptual diversity (e.g., electrical plugs that look similar but function in different countries). Critically, conceptual diversity was the strongest predictor of memorability (R² = .57), and adding perceptual diversity or model-based measures did not improve prediction. Across models, correlations with human conceptual and perceptual judgments both increased with layer depth, and all three architectures converged to similarly good predictions at their final layers. However, none matched the predictive power of human conceptual diversity ratings in predicting human memorability. These findings extend prior work linking conceptual distinctiveness to memory by demonstrating that purely local, within-category structure, independent of a category's position in global semantic space, predicts how well individual exemplars are remembered. The gap between human and model-derived diversity estimates suggests that the typical implementations of vision models lack the flexible, context-sensitive feature weighting that humans deploy when organizing objects conceptually, highlighting an important constraint for computational accounts of human visual memory.

Topic Area: Memory, Learning & Knowledge Structures