Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster D in Poster Session D: Wednesday, August 5, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Interfacing Macaque Cortex with Language Models for Generative Neural Readout

Victoria Bosch¹, Daniel Anthes², Adrien Doerig³, Peter König¹, Tim C Kietzmann¹; ¹Universität Osnabrück, ²University of Osnabrück, ³Freie Universität Berlin

Presenter: Victoria Bosch

Recent advances in neural decoding have moved beyond categorical readouts toward richer, semantically grounded representations, for example by mapping brain activity elicited by natural scenes to language embeddings of image captions. While effective, such approaches remain limited to static representations. To enable more flexible and interactive decoding of neural data, recent work has proposed interfacing brain activity directly with large language models (LLMs), allowing neural signals to condition open-ended language generation. However, these approaches have so far been restricted to human fMRI and time-averaged signals. Here, we extend this approach to intracranial recordings from macaque visual cortex during passive viewing in the THINGS dataset. We show that time-resolved neural activity is decodable into natural language, enabling generative readout of visual content. These preliminary results demonstrate that LLM-based neural decoding interfaces generalize beyond human fMRI and suggest the presence of structure in non-human primate visual representations that can be aligned with natural language embeddings.

Topic Area: Methods, Tools, Theory & Neural Coding