3

I want to create 100% synthetic voice speech with no human voice to make my instrument say and sing lyrics.

I looked at the formants of different vowels like "A" or "E" or "O"

enter image description here

For example, "u" sounds with F1 at 300 Hz and F2 at 800 Hz, so I created a wave and I increased the volume in the 2 formants like this with Audacity:

enter image description here

But we cannot recognise any vowels, Why not? https://voca.ro/1kE9qIH5Yyfk

4
  • 2
    Google “talking piano” for several attempts that came before you. Feb 27 at 20:21
  • Speech synthesis has been accomplished (and become quite advanced—thus Siri and Alexa). It might be educational to try to re-invent this wheel, but maybe research how the pioneers of synthesis handled the problem in the 20th century? Feb 27 at 21:05
  • Yamaha did this starting the late 90s with Vocaloid (and it's now on version 5). vocaloid.com/en Be aware, it's very hard, and it's unlikely you are going to be able to do anything just in Audacity. You will end up creating a very complex VST plugin. Better start reading up about digital signal processing too.
    – Neil
    Feb 28 at 14:13
  • Start by studying how vocoders work. Once you grok that, the path should be obvious. This will require significant signal processing skills.
    – J...
    Feb 28 at 15:47

4 Answers 4

4

The problem with your Audacity experiment is probably that the EQ filters just aren't steep and selective enough. (and using a single sine wave as input, which leaves the filters with nothing to filter) Try something like Pure Data and use three or more band-pass filters in parallel (not in series), creating a really steep contour. Try searching for formant filters, they are done as parallel band pass filters. Your source article gives you frequencies F1 and F2, but you need something that gives you F3 etc. And use band-pass filters in parallel, not Audacity's EQ curves.

I once tried this and I was able to do a semi-intelligible vocal synth with one oscillator (actually, more than one because there was noise as well) and a formant filter consisting of a bank of three filters set to the right frequencies. To switch between vowels, morph/slide the three filters to the next vowel's frequencies. IIRC, two filters was too few, there needs to be at least three filters to create recognizeable vowels. Some letters like W that are classified as consonants can be done by morphing the formant filters between vowels very quickly. Plosives like t, p, k are more difficult and I couldn't do those with the simple synth building blocks I tried, namely oscillators and filters.

Here's a Sound on Sound article on formant synthesis https://www.soundonsound.com/techniques/formant-synthesis

Edit. I didn't look at your audio example. Seeing user1079505's answer, I recalled what the oscillator was in my vocal synth. It was not a simple sine wave, but something more complex like a square wave that has many partial frequencies besides the fundamental - and I mixed in some noise to simulate breathiness. More noise = more breathiness. If you have only noise, then it sounds like whispering. Wind noise of air flowing through a pipe, filtered by a formant filter. But filters need there to be something to filter. If there's no frequency content at, say, 400 Hz, then what does a band-pass filter at 400 Hz let through? Nothing.

2

For example, the "u" sounds with a F1 at 300 Hz and a F2 at 800 Hz, so I created a wave and I up the volume in the 2 formants like this with audacity

This is the wave that you created:

enter image description here

And this is its spectrum:

enter image description here

  1. It appears to me that your wave is a pure sine wave at 100 Hz. It does not contain any other frequencies (apart from distortions, see the next point), so boosting 300 Hz and 800 Hz won't do anything. EQ won't create sound at frequencies missing in the input signal. In order to imitate human voice you need a wave more resembling broad spectrum of human voice, e.g. containing multiple harmonics.

  2. The wave is distorted (close to the left edge of the waveform image), and also saturated (right part of the image). I don't know what part of your process makes this, but it results in audible artifact, that may impede speech sound.

0

There is little use to eq certain frequencies when your base signal is a sine wave. You should start with a signal with wide harmonic spectrum. ZynAddSubFx has a special formant filter for such stuff, if you are interested.

-1

The model seems far too simple; even if you would manage the vowels, the consonants will become insurmountable (due to lack of periodicity).

Either the synthesizer supplies something (see General Midi, programs 53 and 54 out of the box, or you will run out of oscillators. Compared to speech (song) the sound curves of real instruments are simple, and they use recorded samples for any decent approximation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.