Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

NeuroBabyLM: A Benchmark for Brain Alignment in Language Models Trained on Developmentally-Plausible Datasets

Ahhyun Lucy Lee¹, Danny Dongyeop Han¹, Kijeong Sohn¹, Nikunj Agarwal¹, Jiook Cha¹; ¹Seoul National University

Presenter: Ahhyun Lucy Lee

Do language models trained on human-like data develop human-like internal representations? The BabyLM Challenge trains models on child-scale corpora (≤100M words) and evaluates them on behavioral benchmarks—yet behavioral scores cannot confirm internal neural similarity. We introduce NeuroBabyLM, a novel benchmark that extends the BabyLM evaluation suite with fMRI based brain alignment scores. Using ridge regression encoding models on a naturalistic listening fMRI dataset, we evaluate 15 models spanning BabyLM variants, Pythia, and LLaMA across a wide range of magnitude in both size and training data. We find: (1) brain alignment generally increases with model size (14M–65B); (2) T5-Base trained on 100M words outperforms Pythia-160M trained on ∼225B words, showing that the joint combination of architecture, objective and training domain can override raw data scale; and (3) data-scale sensitivity is architecture-dependent. These results reveal a representational alignment gap: models trained on developmentally plausible data volumes do not automatically yield brain-like internal representations. We position NeuroBabyLM as a systematic testbed for probing which architectural and training factors give rise to such representations.

Topic Area: Methods, Tools, Theory & Neural Coding