SPEECHSPLIT2: DISENTANGLING SPEECH INFORMATION STREAMS WITHOUT EXHAUSTIVE BOTTLENECK FINE-TUNING

Permalink

https://hdl.handle.net/2142/124961

Title: SPEECHSPLIT2: DISENTANGLING SPEECH INFORMATION STREAMS WITHOUT EXHAUSTIVE BOTTLENECK FINE-TUNING
Author(s): Chan, Chak Ho
Issue Date: 2021-05-01
Keyword(s): VoiceConversion; Speech Disentanglement; Signal Processing
Abstract: SpeechSplit is among the first algorithms that successfully disentangle speech into four components: rhythm, content, pitch, and timbre. However, the model requires exhaustive fune-tuning of the bottleneck dimensions of the encoders, which can be a daunting task and limits its generalization ability. In this work, we propose SpeechSplit2, an improved version of SpeechSplit, in which simple signal processing methods are utilized to alleviate the laborious bottleneck fine-tuning problem. We show that by feeding di↵erent inputs to each encoder, we can control the input space to the neural networks so that each component only contains the information that we desire to extract, given the bottleneck size is su ciently large to encode the corresponding information. With the same neural network architecture as SpeechSplit, SpeechSplit2 achieves comparable performance in disentangling speech components when the bottlenecks are carefully fine-tuned and shows superior advantage over the baseline when the bottleneck size varies.
Type of Resource: text
Language: eng

The best of ECE undergraduate research