How do polyphonic pitch-shifting algorithms handle doubled/tripled octaves?

Question

I do not in the first place how polyphonic pitch shifters work. What happens internally when one that’s adding octaves (or simply, octaves are being played) is fed into another? Does it treat the lowest note as the fundamental and the multiples as harmonics? I assume pitch-shifters have a way of filtering out harmonics from played notes.

Answer 1

A simple question, but the answer is pretty complex. I will try to squeeze 40 years of research into some sentences.

The first realtime pitch shifters worked with using granular synthesis in the time domain. This means the incoming signal gets chopped into small pieces called windows, resampled at higher or lower speed and finally recombined with soft fading, trying to avoid phase problems and/or transitions cancelling, doubling and delaying. The same family of problems is found in post production time-stretching.

Today's algorithms use mainly frequency domain algorithms, this means decompose the signal (i.e. most known is FFT), meddle with phases and sudden changes, recombine (synthesis) the signal for creating an interpolated output. These algorithms are trimmed to preserve the parts that are important to our ears (the transitions, e.g. pick, nails, blow, percussive, voice consonants). Depending on the algorithms you will have some that are tonally "more" correct and others that will have less problems with latency on transitions. Psycho acoustical studies (i.e. masking of transitions related to speed) are an important part in all these designs.

If you listen carefully or use artificial signals you will note that some signals are not as clean as you think they should be after pitching.

Voice pitching: some pitchers will allow to preserve formants, because they characterise our voice, otherwise a pitched voice sounds more like a chipmunk, Donald Duck style. Therefore auto tuners and voice harmonisers use different algorithms than guitar targeted pedals or postproduction pitch shifter plugins. In post production the program may take their time to analyse the signal and have the best results for studio productions.

So, to your questions: yes and no, some would treat the fundamental as a base to avoid phase cancelling. Others will take care in remixing the windows with maximum correlation.

And a small hint: routing from one pitch shifter into another is usually not a good idea, because the already existing artefacts will increase. That is also a good reason for having a pitch shifter early in the chain of effects, e.g. not after a reverb. But then again, artistically expression doesn't stop us to do the crazy combinations and the ugliness of autotune becomes a trademark for others.

Answer 2

I think there is a key point that the PS-6 emphasizes very well:

• pitch-shifting can be fully polyphonic, basically because I does not care what are the notes played (I'll expand after),
• harmonizer need to produce notes inside a scale. For that they need to recognize the note played at different times in order to produce another one (for instance, minor third or major third, depending on the scale).

Pitch shifters…

… Do not care about the note which is played. They do not need to, because equal temperament is a logarithmic scale: want the octave? Multiply the frequency by 2. Want the fifth? Multiply by 2^(7/12). As the "multiplier" will be the same for all notes, well, the pitch-shifter "just have" to multiply the frequency of the whole signal (I will expand again).

How do they do that?

I do not work in the pitch-shifting industry but here is a basic (lame) idea/algorithm that should work. Let's say you want to shift all your notes by a fifth, all frequencies have to be multiplied by 2^(7/12), eq. all periods have to be divided by 2^(7/12). A way to go is to record a small part of the signal (say, 1ms, no too big for latency, not to small to keep the low frequencies) and then play it 2^(7/12) faster. It will actually divide all the periods (multiply the frequencies) by this amount effectively pitch-shifting your signal without caring of what notes were in there:

• an A 220Hz, will find itself at 220*2^(7/12) = 329.6 = E
• an E 329.6Hz will find itself at 329.6*2^(7/12) = 493.8Hz = B

All cool! Except that, if we play the sample faster, then there is a gap at the end before the next sample comes in. Once again, I am not in the industry but a safe choice would be to replay a part of the accelerated sample to fill the gap.

Obviously, pitch-shifting to lower notes is exactly the same, but playing slower. You then need the cut part of your signal out (too bad) are the sample is now bigger than the original.

So now you understand that pitch-shifter do not care if they are daisy-chained: you just tell them the ratio of frequency you want, and they apply that kind of algorithm to the whole, without even trying to get what notes are in there: they actually work on non-pitched sounds.

A lot of words for a blurry answer I'm afraid, you should not ask things like that before the morning coffee…