Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

Toward a Shared Sensorimotor Language: Affordances in a Global Workspace

Yusuf Helal¹, Rufin VanRullen²; ¹Université de Toulouse-le-Mirail (Toulouse III), ²CNRS

Presenter: Yusuf Helal

Perception and action are deeply intertwined: effective behavior depends on building a common representational language between sensory and motor information. Gibson (1979) argues that biological agents naturally structure perception around affordances (i.e., action possibilities). In contrast, artificial models often rely on modality-specific encodings, limiting their ability to flexibly couple perception and action in complex environments. The Global Workspace Theory (GWT; Baars, 1988) proposes a shared informational space through which cognitive systems broadcast task-relevant signals across functional modules. In this paper, we use a deep-learning architecture leveraging principles from GWT to learn a shared sensorimotor latent space jointly representing vision and action. We analyze the structure of this space, comparing visual representations within our Global Latent Workspace (GLW) to those learned by a (unimodal) variational autoencoder. We show that the GLW can organize visual representations according to action-relevant structures, yielding improved clustering and downstream visual object classification performance. These results provide a pathway towards building sensori-motor systems that are grounded in action possibilities and provide embodied knowledge of the world.

Topic Area: Computational Models of Vision & Visual Cortex