# How do polyphonic pitch-shifting algorithms handle doubled/tripled octaves?

I do not in the first place how polyphonic pitch shifters work. What happens internally when one that’s adding octaves (or simply, octaves are being played) is fed into another? Does it treat the lowest note as the fundamental and the multiples as harmonics? I assume pitch-shifters have a way of filtering out harmonics from played notes.

• What do you exactly mean by " is fed into another"? In another pitch-shifter? So you more interested in the effect-pedal, rather than the pitch-bend in synth for instance? Note that, usual octave pitch shifter usually do not care what is the note played, but rather double everything up using a rectifier for instance.
– Tom
Oct 15, 2020 at 13:53
• I think this is a good question, and I'd like to know the answer, but it's kinda difficult to understand exactly what you mean? For example what does "one that's adding octaves" mean? Could you give an example of a polyphonic algorithm? For autotune, as far as I know, you feed it a monophonic input. I don't know of any polyphonic alternatives.
– Alan
Oct 15, 2020 at 15:04
• Are you looking at an algorithm that will shift the entire signal from a polyphonic source by a set amount (such as zplane elastique, or a digitech drop pedal), or one that will shift the individual notes separately (such as melodyne's "DNA")? Oct 15, 2020 at 21:43
• For argument’s sake imagine a guitar going into a BOSS PS-6 fed into another one. Oct 16, 2020 at 3:45
• "I assume pitch-shifters have a way of filtering out harmonics from played notes": why? I suppose you've never spent much time with a phonograph or an adjustable-speed tape player. Oct 18, 2020 at 21:28

A simple question, but the answer is pretty complex. I will try to squeeze 40 years of research into some sentences.

The first realtime pitch shifters worked with using granular synthesis in the time domain. This means the incoming signal gets chopped into small pieces called windows, resampled at higher or lower speed and finally recombined with soft fading, trying to avoid phase problems and/or transitions cancelling, doubling and delaying. The same family of problems is found in post production time-stretching.

Today's algorithms use mainly frequency domain algorithms, this means decompose the signal (i.e. most known is FFT), meddle with phases and sudden changes, recombine (synthesis) the signal for creating an interpolated output. These algorithms are trimmed to preserve the parts that are important to our ears (the transitions, e.g. pick, nails, blow, percussive, voice consonants). Depending on the algorithms you will have some that are tonally "more" correct and others that will have less problems with latency on transitions. Psycho acoustical studies (i.e. masking of transitions related to speed) are an important part in all these designs.

If you listen carefully or use artificial signals you will note that some signals are not as clean as you think they should be after pitching.

Voice pitching: some pitchers will allow to preserve formants, because they characterise our voice, otherwise a pitched voice sounds more like a chipmunk, Donald Duck style. Therefore auto tuners and voice harmonisers use different algorithms than guitar targeted pedals or postproduction pitch shifter plugins. In post production the program may take their time to analyse the signal and have the best results for studio productions.

So, to your questions: yes and no, some would treat the fundamental as a base to avoid phase cancelling. Others will take care in remixing the windows with maximum correlation.

And a small hint: routing from one pitch shifter into another is usually not a good idea, because the already existing artefacts will increase. That is also a good reason for having a pitch shifter early in the chain of effects, e.g. not after a reverb. But then again, artistically expression doesn't stop us to do the crazy combinations and the ugliness of autotune becomes a trademark for others.

• A good book for someone who likes to dig deeper into Digital Audio Effects is: DAFX by Udo Zölzer (second edition). Oct 21, 2020 at 12:24
• I think the OP was talking about polyphonic note detection and pitch shifting like in Melodyne. And the problem being, if several voices play the same note but in different octaves, how to know that they're not just one voice with harmonics. And how to assign each harmonic in the spectrum to one of the voices so that the timbre is preserved when pitch-shifting individual voices. Like if you have a chord with three notes C3 - C4 - C5 and you want to pitch them to C3 - E4 - B4. I'm not sure if this is possible in Melodyne, but at least it's possible to pitch C4 - E4 - G4 to C4 - Eb4 - G4. Oct 21, 2020 at 17:40

I think there is a key point that the PS-6 emphasizes very well:

• pitch-shifting can be fully polyphonic, basically because I does not care what are the notes played (I'll expand after),
• harmonizer need to produce notes inside a scale. For that they need to recognize the note played at different times in order to produce another one (for instance, minor third or major third, depending on the scale).

### Pitch shifters…

… Do not care about the note which is played. They do not need to, because equal temperament is a logarithmic scale: want the octave? Multiply the frequency by 2. Want the fifth? Multiply by 2^(7/12). As the "multiplier" will be the same for all notes, well, the pitch-shifter "just have" to multiply the frequency of the whole signal (I will expand again).

### How do they do that?

I do not work in the pitch-shifting industry but here is a basic (lame) idea/algorithm that should work. Let's say you want to shift all your notes by a fifth, all frequencies have to be multiplied by 2^(7/12), eq. all periods have to be divided by 2^(7/12). A way to go is to record a small part of the signal (say, 1ms, no too big for latency, not to small to keep the low frequencies) and then play it 2^(7/12) faster. It will actually divide all the periods (multiply the frequencies) by this amount effectively pitch-shifting your signal without caring of what notes were in there:

• an A 220Hz, will find itself at 220*2^(7/12) = 329.6 = E
• an E 329.6Hz will find itself at 329.6*2^(7/12) = 493.8Hz = B

All cool! Except that, if we play the sample faster, then there is a gap at the end before the next sample comes in. Once again, I am not in the industry but a safe choice would be to replay a part of the accelerated sample to fill the gap.

Obviously, pitch-shifting to lower notes is exactly the same, but playing slower. You then need the cut part of your signal out (too bad) are the sample is now bigger than the original.

So now you understand that pitch-shifter do not care if they are daisy-chained: you just tell them the ratio of frequency you want, and they apply that kind of algorithm to the whole, without even trying to get what notes are in there: they actually work on non-pitched sounds.

A lot of words for a blurry answer I'm afraid, you should not ask things like that before the morning coffee…

• The 'gaps' you mentioned are treated with overlapping various buffers and synthesise a new signal (granular (re)synthesis). The rest of your answer: it the algorithms would not care about the treated material they will create nasty artefacts (doubling or cutting transitions, garbling sounds because of phase cancelling). Pitch shifting is a very complex topic and not so easily answered and surely not as trivial as described. In a nutshell: they care about the material to be interpolated. They consider psycho acoustical problems (preserve transitions), latency issues. Search on aes.org Oct 21, 2020 at 8:14
• @Jürgen My whole point was that, whatever algorithm you use (dilating the FFT with enveloppe follower, granular synthesis…), you do not need to seek for the note actually played. Which was the concern of the OP. I added "lame" to qualify my algo to show that it is just a "proof of concept", and not a professional grade solution… As I said: "I do not work in the pitch-shifting industry" ;)
– Tom
Oct 21, 2020 at 9:35
• point taken. I am working in that industry, so my view is pretty biased! From experience I know it is easy to go here down the rabbit hole! XD Oct 21, 2020 at 11:06
• @Jürgen That's always interesting to have the point of view of a pro ;)!
– Tom
Oct 21, 2020 at 11:11