Singing to speech conversion with generative flow

Singing to speech conversion with generative flow

Blog Article

Abstract This paper introduces singing to speech conversion (S2S), a cross-domain voice conversion task, and presents the first deep learning-based S2S system.S2S aims to transform singing into speech while retaining the phonetic information, reducing variations in pitch, rhythm, and timbre.Inspired by the Glow-TTS architecture, the proposed ULT DIGEST ENZ URGENT CARE model is built using generative flow, with an adjusted alignment module between the latent features.

We adapt the original monotonic alignment search (MAS) to the S2S scenario and utilize a duration predictor to deal with the duration differences between the two modalities.Subjective evaluations show that the proposed model outperforms signal processing baselines in Spindle Mixer Parts and Accessories naturalness and outperforms a transcribe-and-synthesize baseline in phonetic similarity to the original singing.We further demonstrate that singing-to-speech could be an effective augmentation method for low-resource lyrics transcription.