2

Is changing pitch enough for anonymizing a person's voice? piqued my interest about digitally altering voices and the similarity to vocoders in music, and it just occurred to me: The carrier signal could be just about anything, right? So what prevents me from doing the exact same process described in the question but using someone else's singing voice as a carrier signal in order to imitate their voice?

This is just a guess but would I have to have a carrier signal that continuously changes in order to mimic their speech, since their formants/timbral information change over time in order to form words? Has this been achieved? If not, what makes it difficult?

My attempt to summarize the process vocoders use:

  • The input signal (my voice) is separated into narrow intervals of frequency by way of the Fast Fourier Transform or something similar.
  • The carrier signal (usually like a synth or something) has a specific pattern of overtones and harmonics that are different from the ones naturally present in the input (my voice)
  • The input signal has every frequency band's amplitude (volume) altered to match the pattern that the carrier established
  • The resulting frequency bands are combined to form the output signal
Improve this question
3
  • Huh, this new 'Ask Question' page is cool. Let me know if I got anything wrong about how vocoders work, or just edit it in yourself if you're so inclined.
    – user45266
    Commented Mar 13, 2020 at 20:00
  • 1
    You do have a typo - "baturally" instead of naturally - in the second bullet. I don't have enough rep to do a single-character edit.
    – dwizum
    Commented Mar 13, 2020 at 20:25
  • 1
    “naturally” is all I can contribute to this question ;)
    – Albrecht Hügli
    Commented Mar 14, 2020 at 15:28

1 Answer 1

Reset to default
2

The big problem is that the source wave form is rather complicated. A singing voice is not something that is easily composed like an instrument sound (and instrument sounds themselves are remarkably complicated which is part of why primitive sampling technologies (like the Mellotron) couldn't effectively emulate real instruments. You would need to have samples for every phoneme in the singer's voice and be able to match the timing of the "vocoded" (because, again, this is a lot more complicated than just having the singer go "ahhhh" into a sampler and running a vocoder against that sampler).

It will likely be possible to synthesize any voice singing any words/melody at some point in the near future, much like the deepfake videos, but it's a non-trivial problem and likely to come after the comparatively simpler but still under-development being able to do natural-sounding speech synthesis from an existing voice.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.