Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

Human and LLM Alignment in Causal Reasoning for Narrative Events

Xian Li¹, Xiyu Li¹, Janice Chen¹; ¹Johns Hopkins University

Presenter: Xian Li

Causal relationships between events in a narrative are essential for comprehension and memory of that story. While large language models (LLMs) can readily calculate semantic relations between events, it is unclear whether they can produce human-like judgments of how distinct events in a narrative are causally related. Here, we compared human and LLM ratings of the causal connections between narrative event pairs. Across multiple models, LLM-derived judgments of causal relations were significantly correlated with human judgments, with Sonnet4.5 consistently performing on par with humans. LLMs showed high test-retest reliability under different temperatures, though inter-trial variability and its sensitivity to temperature settings differed across LLMs; averaging across temperatures consistently yielded the most human-aligned causal structure. Interestingly, LLM-derived causal relations predicted human memory as well as human ratings did. These findings suggest that LLMs can recover human-like causal representations of narrative events; these methods can aid future studies examining the principles of causal reasoning and narrative comprehension in humans and machines.

Topic Area: Decision-Making, Cognitive Control & Event Cognition