Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Neuroscience-Inspired Analyses of Visual Interestingness in Multimodal Transformers

Mathis Immertreu¹, Fitim Abdullahu², Thomas Kinfe^3,4,5, Helmut Grabner⁶, Patrick Krauss⁷, Achim Schilling⁴; ¹Georg-August Universität Göttingen, ²ZHAW - Zürcher Hochschule für Angewandte Wissenschaften, ³University Hospital Mannheim, ⁴Heidelberg University, ⁵BG Clinic Ludwigshafen, ⁶ZHAW Zurich University of Applied Sciences, ⁷University Erlangen-Nuremberg

Presenter: Mathis Immertreu

Visual interest strongly influences human attention and decision-making, yet it remains unclear whether modern multimodal AI systems internally represent what humans find interesting. We investigated this question using the vision-language model Qwen3-VL-8B and a behavioral measure of visual interestingness derived from Flickr user interactions. Using neuroscience-inspired analyses, we found that visual interestingness is reliably decodable from the model’s representations and becomes progressively more structured from vision to language layers. Moreover, multiple independent methods converged on similar representations, indicating a robust internal encoding of visual interestingness despite the absence of explicit supervision. These findings suggest that large multimodal models capture population-level regularities of human attention and preference and highlight the value of neuroscience-inspired approaches for understanding their internal organization.

Topic Area: Methods, Tools, Theory & Neural Coding