Poster Presentation

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

Revealing core representational axes of neural networks through learning dynamics and brain alignment

Zirui Chen¹, Michael Bonner¹; ¹Johns Hopkins University

Presenter: Zirui Chen

Representations in deep neural networks and brains are encoded in activity patterns distributed over large neural populations. However, there is debate over what axes of population space are most fundamental for understanding these representations. Different perspectives range from rotation-invariant theories that treat axes as incidental, to theories arguing that specific axes carry privileged status. Here, we approach this question through the lens of learning dynamics. We propose that if a set of axes captures the natural structure of a representation, it should reveal a strong temporal progression during learning, with top-ranked dimensions emerging early and remaining stable and higher-ranked dimensions emerging late. We trained a ResNet-50 on ImageNet and compared three axis types—principal components (PCs), nonnegative components, and native axes (channels)—and tracked how dimensions emerge across training epochs. We find that PCs exhibit strongly rank-stratified learning trajectories: top PCs converge rapidly within the first 20 epochs, while higher-ranked PCs continue developing until late in training. In contrast, nonnegative and native axes exhibit little rank-stratified structure. We next extended this analysis to brain-aligned dimensions using human fMRI data from the Natural Scenes Dataset, finding that the most brain-aligned dimensions also emerge early. The convergence between PCA rank, learning order, and brain alignment points to shared organizational principles between artificial and biological visual systems, and extends the notion of privileged axes from a static property of trained networks to a dynamic phenomenon rooted in learning.

Topic Area: Methods, Tools, Theory & Neural Coding