Semantic-prosodic correlates of backchannel utterances in human-computer dialog

Chandler, Ryan Lee

Permalink

https://hdl.handle.net/2142/120270 Copy

Description

Title

Semantic-prosodic correlates of backchannel utterances in human-computer dialog

Author(s)

Chandler, Ryan Lee

Issue Date

2023-04-19

Director of Research (if dissertation) or Advisor (if thesis)

• Shih, Chilin
• Hasegawa-Johnson, Mark

Doctoral Committee Chair(s)

• Shih, Chilin
• Hasegawa-Johnson, Mark

Committee Member(s)

• Girju, Roxana
• Dell, Gary
• Hummel, John

Department of Study

Illinois Informatics Institute

Discipline

Informatics

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Prosody
• backchannels
• back channels

Abstract

Backchannels, words like “uh-huh” and “yeah” in English, serve to provide feedback from listeners to speakers without stopping the speaker from continuing their current utterance or holding the floor. While the function of backchannels to communicate agreement, belief, concern, interest, understanding, and acknowledgement has been identified in former research, there has been little research into how the manner of their articulation translates into these perceived meanings. This dissertation examines these connections between prosody and semantics. An audio corpus of the backchannel utterances “uh-huh” and “yeah” has been constructed to support this research. The corpus features voice actors purposefully inflecting backchannels across a range of semantic dimensions. Acoustic features of pitch, duration, energy, and the pitch energy synchrony are extracted from these samples. A set of predictive models based on multiple linear regression is developed. Significant features are then used to build a minimalist synthesis model. Synthetic backchannels are created using this synthesis model and are then evaluated by other human subjects. In addition to synthesis, a proposed prosodic sub-language is investigated, and several rules of backchannel prosody are proposed that may align to and be grounded in existing research in the field of embodied cognition. The research identifies key features of backchannel prosody that act as relatively strong semantic correlates. And, that these featural guideposts can be modeled and modulated in such a way as to reliably encode semantic intent into the sample backchannels through prosody alone.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Ryan Lee Chandler

Owning Collections