Given a clean WAV recording, of a piano, and sometimes one or 2 string instruments (for example), and a MIDI file of that recording (for that example)

I would like to align the MIDI and the WAV recordings.

I wrote a simple script that:

  1. Detects when the MIDI starts (simple, looks for first "not on" event)
  2. Detects when audio starts (not so simple using heuristic: audio starts when the first sample gets above the mean of all absolute value of samples)

This kind of works, and gets me to 0-500ms delay, depends on the video. To correct that max 500ms delay I generated an audio file from the MIDI recording, and tried to use cross-correlation phase alignment, but its behavior is erratic and it doesn't work as expected.

Is there any tool/software/idea how to align MIDI and WAV files?

  • Aren't there any timing flags in the metadata for such files? – Carl Witthoft Aug 28 '19 at 12:33
  • @CarlWitthoft Unfortunately, both are independently recorded. Think of a person pressing two record buttons, one after the other, with a few seconds delay. – Amit Aug 28 '19 at 12:34
  • 1
    The task challenge might be different if you've a lot of songs to line up, but this is one task I'd just do by eye & ear. It's drag/listen, rinse/repeat. You'll get it in about 2 or 3 attempts; far quicker than trying to design an algorithm that's smart enough to know precisely when a string player is feeling the centre of the beat. Two percussive instruments, fine, but not strings. They don't have enough attack for that type of task. – Tetsujin Aug 28 '19 at 13:58
  • 1
    Welcome to the site Amit. Unfortunately, asking for software recommendations is off-topic here. You may be able to ask on the Software Recs SE if you can format your question to their standards. To find out more about this site take the tour and read the FAQ. – Dom Aug 28 '19 at 15:28

Browse other questions tagged or ask your own question.