Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster C in Poster Session C: Wednesday, August 5, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Mechanisms of Emergent Analogical Mapping

Caleb Musfeldt¹, Taylor Whittington Webb¹; ¹Université de Montréal

Presenter: Caleb Musfeldt

Large language models (LLMs) have become increasingly capable of complex cognitive tasks, including analogical reasoning. Recent mechanistic interpretability work has revealed that these models implement emergent symbolic mechanisms to support abstract reasoning. Cognitive theories have long argued that mapping over structured symbolic representations is a centrally important process for human analogical reasoning. However, whether and how LLMs perform analogical mapping remains an open question. Here, we apply causal mediation analysis to Llama-3.1-70B to identify the mechanisms responsible for analogical mapping. We find a series of three functionally distinct computations implemented by specialized attention heads: (1) Analogical mapping heads align entities across analogs based on shared relational roles; (2) Analogical inference heads use the inferred mapping to determine the correct role for a query item; and (3) Analogical retrieval heads retrieve the specific token associated with the predicted role. These results provide empirical evidence that language models perform analogical reasoning via an emergent version of the mapping process found in cognitive theories of analogy, and establish a foundation for further mechanistic interpretability work on analogical reasoning.

Topic Area: Auditory, Speech & Language Processing