Beyond Voice Identity Conversion: Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations

Voice conversion (VC) consists of digitally altering the voice of an\nindividual to manipulate part of its content, primarily its identity, while\nmaintaining the rest unchanged. Research in neural VC has accomplished\nconsiderable breakthroughs with the capacity to falsify a voice identity using\na small amount of data with a highly realistic rendering. This paper goes\nbeyond voice identity and presents a neural architecture that allows the\nmanipulation of voice attributes (e.g., gender and age). Leveraging the latest\nadvances on adversarial learning of structured speech representation, a novel\nstructured neural network is proposed in which multiple auto-encoders are used\nto encode speech as a set of idealistically independent linguistic and\nextra-linguistic representations, which are learned adversariarly and can be\nmanipulated during VC. Moreover, the proposed architecture is time-synchronized\nso that the original voice timing is preserved during conversion which allows\nlip-sync applications. Applied to voice gender conversion on the real-world\nVCTK dataset, our proposed architecture can learn successfully\ngender-independent representation and convert the voice gender with a very high\nefficiency and naturalness.\n