Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

Large Language Models Infer Hidden Preferences from Gaze but Fail to Exploit Response Times

Mrugsen Nagsen Gopnarayan¹, Liis Harjo², Jaan Aru^3,2, Sebastian Gluth¹; ¹Universität Hamburg, ²University of Tartu, ³institute of computer science

Presenter: Mrugsen Nagsen Gopnarayan

Large language models (LLMs) are increasingly deployed in interactive settings, in which systems must infer a user's preferences not only from explicit instructions but from observed behaviour. To test this, we evaluated three LLMs (llama 3.1, the cognition-tuned Centaur, and GPT-5) as interaction partners in a multi-round negotiation task where human preferences must be inferred from choices, gaze, and response times (RT). Despite lacking cognition-specific training, the frontier model substantially outperformed both open-source models, suggesting that increasing scale and adding reasoning contributes more to preference inference than behavioural fine-tuning. However, none of the models exploited RT as effectively as humans did, highlighting a gap in LLM social cognition. Furthermore, humans remained superior at recovering true preference weights, though models translated imperfect estimates into competitive offers, pointing to their better quantitative abilities. Traces of decision processes therefore help LLMs only selectively: Whereas gaze improves preference inference, RT are so far of limited use.

Topic Area: Decision-Making, Cognitive Control & Event Cognition