Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

A Parametric Continuum for Probing Visual Speech Representations

Emma Zhang¹, Yike Li¹, David Brang¹; ¹University of Michigan - Ann Arbor

Presenter: Emma Zhang

Understanding speech in noise is substantially improved by seeing the speaker's face. However, seeing DNN-generated synthetic talking faces provides weaker benefits than seeing real talking faces, and it remains unclear which specific facial movements drive this audiovisual gain. Here, we leverage DNN-based synthetic faces to achieve precise control over visual speech components. Using DECA talking face animations, we constructed a synthetic continuum that systematically manipulates specific expression parameters one at a time and validated this approach psychophysically with human observers. This framework serves as a first step toward a computationally tractable approach to probing cognitive representations of visual speech, enabling identification and quantification of diagnostic facial components that enhance phoneme discriminability.

Topic Area: Auditory, Speech & Language Processing