University of Illinois Urbana-Champaign

SpeechSplit2: Disentangling Speech Information Streams without Exhaustive Bottleneck Fine-tuning

Chan, Chak Ho

This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.

Permalink

https://hdl.handle.net/2142/110321

Description

Title
SpeechSplit2: Disentangling Speech Information Streams without Exhaustive Bottleneck Fine-tuning
Author(s)
Chan, Chak Ho
Contributor(s)
Hasegawa-Johnson, Mark
Issue Date
2021-05
Keyword(s)
  • Voice Conversion
  • Speech Disentanglement
  • Signal Processing
Abstract
SpeechSplit is among the first algorithms that successfully disentangle speech into four components: rhythm, content, pitch, and timbre. However, the model requires exhaustive fine-tuning of the bottleneck dimensions of the encoders, which can be a daunting task and limits its generalization ability. In this work, we propose SpeechSplit2, an improved version of SpeechSplit, in which simple signal processing methods are utilized to alleviate the laborious bottleneck fine-tuning problem. We show that by feeding different inputs to each encoder, we can control the input space to the neural networks so that each component only contains the information that we desire to extract, given the bottleneck size is sufficiently large to encode the corresponding information. With the same neural network architecture as SpeechSplit, SpeechSplit2 achieves comparable performance in disentangling speech components when the bottlenecks are carefully fine-tuned and shows superior advantage over the baseline when the bottleneck size varies.
Type of Resource
text
Language
en
Permalink
http://hdl.handle.net/2142/110321

Owning Collections

Senior Theses - Electrical and Computer Engineering PRIMARY
The best of ECE undergraduate research