Is converting audio to MIDI or sheet music difficult (for pro musicians or software)?

Question

I watched this Youtube video about converting audio to MIDI, and the comments mostly point out that it's a scam or hoax.

Is it really difficult for software to convert audio to MIDI, and if so why?

Also, do pro musicians find this hard too?

Answer 1

Yes, it's hard. And it gets harder as the music gets more complex. I think there would be general agreement that, at the present state of the art, a skilled and experienced musician is way ahead of any computer program. But computers have a habit of catching up....

Answer 2

I had a quick listen to part of the YouTube video - they guy is confusing audio & MIDI a bit, really. He's done the multi-pitch analysis, & is then dragging notes up & down... fine, but that's not MIDI, that's Melodyne live-shifting one single note from the chord as audio, in realtime [which is one hell of a feat anyway]

To the uninitiated - Melodyne is "Auto Tune" on steroids.

It is neither scam nor hoax. I own the software & have done for a decade [or more] The manufacturer, Celemony, have a scant entry on Wikipedia but you can investigate Melodyne on their own site.

Melodyne is about the only commercially-available software than can analyse chords & separate the pitches. With audio, you can then shift pitches within the file without changing the entire song.

Whether it can convert this, without error, into MIDI would be a whole different level of task, & one I have never actually needed to test. I use it entirely as an audio medium.

What it can do is such as this...

REM - Losing My Religion in a major key

Yes, it sounds truly weird :\ but if you didn't know the original, I doubt you would be able to tell what had been done to it.

Pitch tracking itself has come a long way over the years.
Back in the late 90's I worked for Yamaha R&D & we were truly proud at that time to be able to play midi drums & piano, whilst live-tracking a real bass guitar, audio to midi, & sync that all to a streamed vocal recording, with a video of the entire performance, as a kind of 'simulcast'.
It's too old to have ever made it to YouTube, so I can't demonstrate how it did actually work - even in a rather primitive way.

Do pro musicians find this hard too?
Some do, some don't. I know many people who can transcribe accurately with little repetition, though mostly they listen to one instrument at a time.
I do know one guy who can transcribe an orchestra, vertically, down the page, in 2-bar segments, pausing the audio only long enough for him to physically write all the notes.
That is an extremely rate talent.

I realised I have one of his transcriptions already on Soundcloud - this is a MIDI file, a "one box demo" i.e. all played at once into a single 16-channel MIDI device (Yamaha XG MU100, circa 1997), the stereo output recorded untouched. I did the mix, he did the transcription from audio to MIDI.