Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Evaluating EMEG and ECoG Modalities for Brain-LLM Alignment in Speech Perception

Chentianyi Yang¹, Andrew Thwaites², Alexandra Woolgar¹; ¹University of Cambridge, ²University College London

Presenter: Chentianyi Yang

Large Language Models (LLMs) are increasingly used as models for speech perception, yet it remains unclear if brain-LLM alignment is consistent across recording modalities. In this study, we compared the alignment of an audio LLM with the neural activity from electro-magnetoencephalography (EMEG) and electrocorticography (ECoG). Using a unified pipeline, we mapped single-neuron activations across all LLM layers to neural responses to naturalistic speech. Our results demonstrate a convergence in temporal hierarchy: both modalities show a strong linear correlation between layer depth and response latency, suggesting a modality-independent processing map. We also observed divergence in feature sensitivity: ECoG captures more significant neurons in middle layers, while EMEG provides better matches at deeper layers. These findings suggest that the hierarchy of speech processing is preserved across modalities, but fine-grained features can differ. Our study provides a benchmark for multi-modal alignment and highlights the importance of single-neuron analysis.

Topic Area: Auditory, Speech & Language Processing