Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster C in Poster Session C: Wednesday, August 5, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Effective dimensionality tracks the time-varying accessibility of input features during speech comprehension

Emily Cheng¹, Christopher Wang², Andrei Barbu³, Marco Baroni¹, Greta Tuckute⁴; ¹Universitat Pompeu Fabra, ²Computer Science and Artificial Intelligence Laboratory, Electrical Engineering & Computer Science, ³Amazon, ⁴Harvard University

Presenter: Emily Cheng

How does the brain dynamically build rich meanings from acoustic information during spoken language understanding? Using an intracranial electroencephalography (iEEG) dataset from ten participants (1,688 electrodes) watching movies with naturalistic dialogue, we asked how the evolving geometry of the neural high-gamma signal relates to speech processing at sub-word temporal resolution. We found that the effective dimension of the high-gamma response at individual electrodes expands and compresses rapidly over the time course of a single word, and critically, that transient expansions in dimensionality track the availability of linearly decodable speech and linguistic information about the input---a signature predicted by theoretical neuroscience and experimentally confirmed for the first time in the domain of speech comprehension. The link between dimensionality and linear decodability of input information is further shown by the fact that dimensionality was generally higher during speech than non-speech. Taken together, our results show that the dimensionality of the neural high-gamma signal provides a temporally precise marker for when input-relevant features become linearly accessible during naturalistic speech comprehension.

Topic Area: Auditory, Speech & Language Processing