Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster B in Poster Session B: Tuesday, August 4, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Tracing Sparse Feature Circuits in Vision Transformers

Zirui Hu¹; ¹Brown University

Presenter: Zirui Hu

This work implements a pipeline for cross-layer circuit analysis in Vision Transformers (ViTs), combining sparse feature learning, candidate filtering, relation analysis, and intervention-based tests. Sparse autoencoders are trained on intermediate activations, candidate units are filtered using statistical and visual criteria, and retained units are analyzed through cross-layer consistency, feature-to-output edge strength, preliminary feature-to-feature relations, and targeted interventions. The current results already produce units and local structures that can be used for early circuit analysis in ViTs, and provide a workable basis for broader cross-layer circuit reconstruction.

Topic Area: Methods, Tools, Theory & Neural Coding