Think about this from a computing perspective.
Voice is your built-in default output, optimized from the ground up for power-efficiency and instruction-cycle efficiency in the operating system and hardware.
Now consider replicating music on an instrument. An instrument is external hardware that was never optimized in the operating system or hardware it's being run on. In the computing world, using new peripherals either involves loading drivers (plug-ins at the operating system level) or using a program that comes with the peripherals (application-layer interface).
People with perfect pitch could be compared to having efficient operating-system level drivers for the new peripheral (instrument). Good, although slower than optimization in hardware.
Everyone else (with no perfect pitch) could be compared to using an application-layer interface (specific program) to control the peripheral (instrument). Slow and worst performance compared to OS-level drivers or hardware optimization.
Alternately consider a data-oriented perspective.
Audio input is fundamentally analog, and for purpose of discussion assume that the brain stores audio input in a simple analog format (similar to a WAV file on computer). Voice output natively accepts the same analog format, which means there's almost no processing overhead in echoing what you just heard.
Think of musical notes as a digital (logical) format - a sequence of note begin/end markers. To play an instrument, you have to run through the following algorithm. For purposes of simplicity, assume that you're only playing 1 instrument at a time.
instrument D = getCurrentInstrument().open()
P = NoteInputStream(AnalogAudioInputStream(input song), D)
while true:
note N = P.nextNoteMarker()
if N == null then exit
else if N.isBegin then D.holdNoteAfterDelay(N, N.delay)
else if N.isEnd then D.releaseNoteAfterDelay(N, N.delay)
Seems simple enough in pseudocode. But what's really going on? The complexity is in the API functions above.
getCurrentInstrument() is a function that returns a reference to the instrument in your hands. Relatively simple. The instrument.open() function is another thing entirely. This loads the drivers for the instrument - in other terms, you have to "recall" how to play the instrument. This includes mentally mapping logical notes to whatever you have to do on the instrument to produce the note (probably a reverse lookup table). If the instrument includes holding and releasing a lot of notes at the same time (like a piano), this also includes figuring out how to optimize the fingers for holding down a set of notes. Therefore, instrument.open() takes a lot of processing power and short-term memory (RAM).
AnalogAudioInputStream(audioInput) takes analog audio input and converts it into a "stream" of data.
NoteInputStream(analogInput, instrument) takes the above-mentioned analog audio stream and an instrument, extracting the notes for the specified instrument from the analog input. This step is where perfect pitch comes in, identifying the "main" instrument and mapping its frequency to logical notes in real-time.
Each note is represented as a key, octave, delay before processing, and begin/end flag that represents if the note represents a hold-down or stop-holding-down event.
Playing back the analog input over instrument: loop over the set of extracted nodes and hold/release on the instrument as dictated. Complexity of hold/release note operations depends widely on the instrument and obviously may fail if you run out of fingers or the input is ridiculous.