Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster B in Poster Session B: Tuesday, August 4, 2:00 – 3:45 pm, Kimmel Center, Shorin & Rosenthal Rooms

Encodings of multimodal conversational behavior predict trait impressions

Landry Bulls¹, Mark A Thornton¹; ¹Dartmouth College

Presenter: Landry Bulls

Stylized mannerisms during social interaction make people's personalities visible to others. However, the dynamic, fine-grained, and multimodal nature of these signals makes quantifying their role in shaping trait impressions difficult to determine. We present an approach for decomposing multimodal social signals—dynamic facial expression, body gesture, vocal prosody, and speech semantics—into translation-invariant encodings of conversational style using the wavelet scattering algorithm. From audiovisual recordings of in-person group conversations (N=120), we extracted multivariate time series for each modality and decomposed them into scattering coefficients weighted by inter-participant discriminability. Using representational similarity analysis (RSA), we found that people with more similar expressive styles were rated as having more similar personalities on the Big-5 Personality Index (BFI), with different modalities contributing differently to each dimension. These results lay groundwork for understanding how the brain exploits statistical regularities in social signals to derive stable personality representations.

Topic Area: Methods, Tools, Theory & Neural Coding