5

I'm working on a project using the MusicNet dataset, and I've encountered some difficulty understanding the timing information provided in the dataset. I've managed to decipher the instrument, note, but I'm struggling with interpreting the timing information. The dataset includes a snippet of a song, and I've shown this section.

start_time end_time instrument note start_beat end_beat note_value
9182 90078 43 53 4 1.5 Dotted Quarter
9182 33758 42 65 4 0.5 Eighth
9182 62430 1 69 4 1 Quarter
9182 202206 44 41 4 3.5 Whole
9182 62430 1 81 4 1 Quarter
33758 62430 42 60 4.5 0.5 Eighth
62430 90078 42 69 5 0.5 Eighth
62430 119774 1 84 5 1 Quarter
62430 119774 1 72 5 1 Quarter
90078 119774 42 65 5.5 0.5 Eighth
90078 119774 43 57 5.5 0.5 Eighth
119774 145886 1 74 6 0.5 Eighth
119774 145886 42 62 6 0.5 Eighth
119774 145886 43 58 6 0.5 Eighth
119774 145886 1 62 6 0.5 Eighth
145886 172510 43 55 6.5 0.5 Eighth
145886 172510 1 76 6.5 0.5 Eighth
145886 172510 42 58 6.5 0.5 Eighth
145886 172510 1 64 6.5 0.5 Eighth
172510 202206 42 57 7 0.5 Eighth
172510 175582 1 65 7 0.0625 Sixty Fourth
172510 175582 1 77 7 0.0625 Sixty Fourth
172510 202206 43 53 7 0.5 Eighth
175582 180190 1 67 7.0625 0.0625 Sixty Fourth
175582 180190 1 79 7.0625 0.0625 Sixty Fourth
180190 183262 1 65 7.125 0.0625 Sixty Fourth
180190 183262 1 77 7.125 0.0625 Sixty Fourth
183262 187870 1 67 7.1875 0.0625 Sixty Fourth
183262 187870 1 79 7.1875 0.0625 Sixty Fourth
187870 193502 1 77 7.25 0.0625 Sixty Fourth
187870 193502 1 65 7.25 0.0625 Sixty Fourth
193502 196574 1 67 7.3125 0.0625 Sixty Fourth
193502 196574 1 79 7.3125 0.0625 Sixty Fourth
196574 199646 1 77 7.375 0.0625 Sixty Fourth
196574 199646 1 65 7.375 0.0625 Sixty Fourth
199646 202206 1 79 7.4375 0.0625 Sixty Fourth
199646 202206 1 67 7.4375 0.0625 Sixty Fourth
202206 204766 1 65 7.5 0.0625 Sixty Fourth
202206 204766 1 77 7.5 0.0625 Sixty Fourth
202206 224734 42 60 7.5 0.5 Eighth
204766 208349 1 67 7.5625 0.0625 Sixty Fourth
204766 208349 1 79 7.5625 0.0625 Sixty Fourth

Could someone please explain how the timing information in this dataset is structured, particularly in relation to the start and end beat columns? Additionally, any insights on how to interpret this timing information would be greatly appreciated. For reference, here's the MusicNet dataset on Kaggle.

I tried to understand the timing information provided in the MusicNet dataset by examining the columns related to timing, such as start_time and end_time. However, I found it challenging to grasp the precise structure and meaning of this timing data. I expected to gain clarity on how the timing information is formatted and how it relates to the musical notes and beats within the dataset.

2
  • 4
    Just because it’s a music stack doesn’t mean understanding a music-related ML dataset is on topic. Seems like you’d do much better looking at the Kaggle forums, trying to contact the author of the dataset, or simply deciding it’s a bad dataset and you shouldn’t use it. Apr 24 at 12:10
  • 5
    I notice that you’ve edited out a sentence from your post that looks suspiciously like GenAI content. Our site policy is that all GenAI content must be properly referenced, see What is this site’s policy on content generated by generative artificial intelligence tools?. Please edit your question to make it clear which parts were generated by AI. Apr 24 at 20:07

4 Answers 4

5

The Kaggle link says that that info has been “acquired from musical scores aligned to recordings by dynamic time warping”.  So those timings are probably from a real recording, giving the points at which their software thinks the notes started and stopped.

Just looking at that data, I'd guess that the important columns are the start_time and end_time, which probably give the exact start and end times in some unit.  (It seems that a quarter note/crotchet is very roughly 50000, so the unit might be of the order of 1/50000 of a second, depending on the tempo.)

The time values are fairly fine-grained.  (They're all 34 less than a multiple of 512, but there are no bigger common factors in the start times or lengths.)  This makes them look like they're from a real human performance (not generated by a sequencer or from a score, and not quantised).  However, many of the notes are in groups of 2–4 starting at the exact same time, which would be extremely unlikely from a human performance.  So that suggests that it's not a precise representation of a human performance (or has been edited).

Either way, I'd guess that the start_beat, end_beat, and note_value are calculated from those raw times.  So those other time-related columns are clearly only a rough approximation to the much more fine-grained timings, and may not be that helpful.

Ultimately, there may not be that much structure in the dataset!  Whatever tool MusicNet is using to identify the note timings may not be very accurate.  Also, most human performances are far from metronomic, so you wouldn't expect each beat to be the exact same length, nor for each note to start and end precisely on a beat (or fraction thereof).  (If you want that, you'd presumably be better off using a score instead.)

If you want to understand what you've got there, I'd suggest finding some way to listen to it.  Perhaps you can convert it into MIDI data or something else you can import into a sequencer and play back?  Your ears will tell you far more than raw numbers.  Failing that, if you can plot them on a graph or chart (perhaps in piano-roll style), then your eyes may be able to tell you a little more.

5

The dataset includes wav audio files. The data you posted seem to come from train data example 1727, and the corresponding audio file, 1727.wav has sample rate of 44.1kHz, which is audio CD standard.

I would assume the start_time and end_time refer to the sample number in the audio file.

See the example below with the selected region showing what apparently is the first piano note (instrument 1, actually two simultaneous notes 69 and 81, an octave apart).

enter image description here

Note also, the onset of the note seems to be labeled a tiny bit too early.

4

Summary of findings

  • The start_beat column represents the beat # of the note from the score except for the strange reason that the earliest note starts at beat #4 instead of beat #1. So you need to add -3 to all values in this column for the actual beat # in the score. For example, measure #2 starts at beat #4 (since the time signature is 3/4), but it will be shown in the start_beat column as #7.
  • The end_beat and the note_value are mostly one-to-one equivalents (but I found anomalies, maybe in the case of ties such as the first note of the contrabass (instrument 44) has end_beat of 3.5 but the note_value is Whole). Again, they are based on the score, not performance. Example: the MIDI note 57 (A3) for MIDI instrument 42 (Viola) on start_beat 7 is an eighth note, matching the score. In that beat you have a lot of sixtyfourth notes for the piano (MIDI instrument 1) since the score shows a trill. First 3 measures of the score
  • The start_time and end_time columns are based on the performance, the time that the note is heard on the WAV file (which is formatted as mono 44,100 Hz sampling). The unit is sample#, thus confirming @user1079505's finding. Example: using the same Viola note above, dividing 172510 and 202206 by 44100 denotes 3.911 and 4.585 second mark, which you can check with a digital WAV editor that can accept samples as unit, such as Audacity. When I play that segment (an Eighth duration), I hear all the notes in the first half of the first beat of measure 2 in the score (piano, viola, cello, and contrabass). Beat 7

How I investigated

I downloaded the dataset and analyze recording #1727 that @user1079505 pointed out. As the dataset description mentions, this is a REAL (human) performance, and as a musician I can tell this is a professional level performance.

  1. Looking at the MIDI file name (1727_schubert_op114_2.mid) gives me the clue that the recording is the 2nd movement of the famous Schubert's Trout Quintet, a Piano Quintet in A Major D667. See a performance with the score in video here. A PDF of the score itself (Breitkopf & Härtel 1886, courtesy of IMSLP) can be read here (2nd movement starts on page 17).

  2. Matching the score with the CSV file (train_labels/1727.csv) using Excel's Data->Filter, it's quite straightforward to see that the instrument column follows the standard General MIDI instrument numbering:

    • #1 is Piano (4th and 5th staff)
    • #41 is Violin (1st staff)
    • #42 is Viola (2nd staff)
    • #43 is Cello (3rd staff, upper voice)
    • #44 is Double Bass (3rd staff, lower voice)
  3. Now that I can isolate the Viola part using Excel Data->Filter on 42 (the easiest to associate since the Viola's part is all eight notes for the first 7 measures), I can isolate the rows for the Viola's first note of each measure and add 2 columns to the CSV: one for the measure number, the other is for measure duration. We can see that the measure duration is relatively constant BUT not the same, thus reflecting a human real time performance (i.e. not mechanical).

start_time end_time instrument note start_beat end_beat note_value m_no m_duration
9182 33758 42 65 4 0.5 Eighth 1 163328
172510 202206 42 57 7 0.5 Eighth 2 161792
334302 359902 42 57 10 0.5 Eighth 3 164352
498654 525278 42 53 13 0.5 Eighth 4 172032
670686 698846 42 65 16 0.5 Eighth 5 162816
833502 863197 42 57 19 0.5 Eighth 6 166912
1000414 1023966 42 57 22 0.5 Eighth 7 158208
1158622 1203166 42 64 25 0.75 Dotted Eighth 8 176640
  1. Opening the Audio WAV file (train_data/1727.wav) using a digital audio editor such as Audacity I try to figure out the unit in the start_time and end_time columns by experimenting with various units that Acoustica offers such as "milliseconds", "samples", "film frames (24 fps)", "CDDA frames (75 fps)", etc.

    It turns out the unit for the start_time and end_time columns is SAMPLE #. Since the recording is in 44100 Hz wav file format, there are 44,100 samples per second. That means measure 7 starts at sample #1000414 (22.685 seconds mark), and ends at sample #1158621 (26.272 seconds mark). I test the theory by playing one measure at a time between Measure 1 and Measure 8. What I hear matches what's in the score !

    Deciphering the start_beat column, I don't know why the note at the first beat of the score is assigned #4, but at least it is consistent with the time signature of the score, which is 3/4. Thus measure 2 starts at beat #7, measure 3 starts at beat #10, etc. and the note_value column matches the score as well.

2

I can only work out backwards from ordinary musical knowledge. "Beat" must be defined to mean "quarter/crotchet"; and I guess all timings are measured in quarters/crotchets. Presumably "start_beat" means the timing of the start of the beat, but "end_beat" appears to be a misnomer, because fairly obviously it means the duration of the note, not the timing (in "beats") of the end of the note. (A "dotted quarter" means 1.5 beats in this system.)

Everybody knows by now that computers only understand (American) English, but I expect this system was designed by someone who understands English even less than a computer does.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.