Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster F in Poster Session F: Thursday, August 6, 1:45 – 3:30 pm, Kimmel Center, Shorin & Rosenthal Rooms

Fine-Grained Emotion Structure in the Brain-Aligned Subspace of a Self-Supervised Video Model

Seokjin Moon¹, Taeyang Lee¹, Jiook Cha¹; ¹Seoul National University

Presenter: Seokjin Moon

Does the human brain share representational structure for emotion with self-supervised video models? Prior work has shown that neural responses to emotional videos are better explained by fine-grained emotion categories than by canonical affective dimensions such as valence and arousal. We ask whether this categorical structure extends to the subspace shared between the human brain and a video foundation model trained without emotion labels. By aligning Brain-JEPA representations of fMRI responses with V-JEPA2 video embeddings, we identify a compact brain-aligned subspace spanning only three leading principal components of the 1,408-dimensional video embedding. Within this shared subspace, emotional-video geometry aligns more strongly with discrete emotion categories than with valence-arousal dimensions (category/V-A R² ratio = 1.44 vs. 1.26 in the full video model space), and this category advantage is stable across all five subjects. These findings suggest that the visually shared brain-model subspace preferentially preserves category-like emotion structure, whereas dimensional affective information may rely more strongly on brain components not captured by this visual model-aligned subspace.

Topic Area: Methods, Tools, Theory & Neural Coding