Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster B in Poster Session B: Tuesday, August 4, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Parameter symmetries determine representational geometry in overparameterized nonlinear networks

Marvin Theiss¹, Lukas Braun², Andrew M Saxe³, Erin Grant⁴; ¹University Tübingen, ²Allen Institute for Neural Dynamics, ³University College London, ⁴University of Alberta

Presenter: Erin Grant

Representations are routinely used in machine learning, psychology, and neuroscience to probe the computations of biological and artificial systems. Yet it remains unclear to what extent computation constrains representation in artificial neural networks. One key obstacle is that neural networks admit parameter symmetries: transformations of the parameters that preserve function exactly while reshaping representational geometry. Here we show that known parameter symmetries act on representations through just three primitives: addition, duplication, and scaling. This yields a closed-form descriptor of representational geometry as a sum of task-linked features and symmetry-induced noise, extending recent dissociation results from linear to nonlinear networks. Overall, our results delineate when representation can support inferences about computation, and when it cannot.

Topic Area: Methods, Tools, Theory & Neural Coding