1

I was wondering, is it possible to work out the equation of a melody from looking at the soundwave (although I don't know how one could obtain it from the recording)? How hard would this be? What mathematical methods and software would be used to do it?

Thank you!

UPD: Specifically, I want to try and work out the equation of a 12-second piano melody. Does using the Fourier Transform for that purpose sound any realistic?

Also, I’m very grateful for the thorough answers you all have provided!

2
  • 4
    What do you consider to be the "equation" of a melody & also what do you consider a "soundwave"? These are not terms usually used to describe music or audio. – Tetsujin Nov 16 '20 at 19:59
  • If by "equation of a melody" you mean transcription or sheet music, them heck, you don't even need the sound wave. Decent transcribers can get the melody of pretty much any recording down, often with the help of their voice, an instrument, and/or the ability to slow the recording down. Really good transcribers can transcribe most to all of the instruments in that recording. – Dekkadeci Nov 17 '20 at 12:59
2

Short answer: You may be interested in the chromagram and/or the Constant-Q transform.

From the links cited above:

Chromagram:

In music, the term chroma feature or chromagram closely relates to the twelve different pitch classes. Chroma-based features, which are also referred to as "pitch class profiles", are a powerful tool for analyzing music whose pitches can be meaningfully categorized (often into twelve categories) and whose tuning approximates to the equal-tempered scale. One main property of chroma features is that they capture harmonic and melodic characteristics of music, while being robust to changes in timbre and instrumentation.

Constant-Q transform

The transform exhibits a reduction in frequency resolution with higher frequency bins, which is desirable for auditory applications. The transform mirrors the human auditory system, whereby at lower-frequencies spectral resolution is better, whereas temporal resolution improves at higher frequencies. At the bottom of the piano scale (about 30 Hz), a difference of 1 semitone is a difference of approximately 1.5 Hz, whereas at the top of the musical scale (about 5 kHz), a difference of 1 semitone is a difference of approximately 200 Hz.[7] So for musical data the exponential frequency resolution of constant-Q transform is ideal.

In addition, the harmonics of musical notes form a pattern characteristic of the timbre of the instrument in this transform. Assuming the same relative strengths of each harmonic, as the fundamental frequency changes, the relative position of these harmonics remains constant. This can make identification of instruments much easier. The constant Q transform can also be used for automatic recognition of musical keys based on accumulated chroma content.[8]

Relative to the Fourier transform, implementation of this transform is more tricky. This is due to the varying number of samples used in the calculation of each frequency bin, which also affects the length of any windowing function implemented.


Some more data

I think what you're looking for is the Fourier Transform.

In mathematics, a Fourier transform (FT) is a mathematical transform that decomposes a function (often a function of time, or a signal) into its constituent frequencies, such as the expression of a musical chord in terms of the volumes and frequencies of its constituent notes. The term Fourier transform refers to both the frequency domain representation and the mathematical operation that associates the frequency domain representation to a function of time.

Specifically, you'd be working with the STFT (short time Fourier transform).

The Short-time Fourier transform (STFT), is a Fourier-related transform used to determine the sinusoidal frequency and phase content of local sections of a signal as it changes over time.[1] In practice, the procedure for computing STFTs is to divide a longer time signal into shorter segments of equal length and then compute the Fourier transform separately on each shorter segment. This reveals the Fourier spectrum on each shorter segment. One then usually plots the changing spectra as a function of time, known as a spectrogram or waterfall plot.

Those are the building blocks. Nevertheless, depending on what you need specifically, you'd probably want to dive into Music Information Retrieval (MIR).

Here there are introductory notes, in which basic concepts are presented.

Here is an article presenting a method of melody extraction, but it is more advanced.

I think these are the building blocks you need to understand to start thinking about the question you brought.

If you have further questions, I suspect they may be more thechnical, hence I'd recommend diving into some other stackExchange sites also: you may find some answers there, too.


As per the software, you may want to start with Librosa, but it's not the only option.

1

An equation is about says whether two things are equal. So, in principle, is it possible to ask:

Is melody #1 equal to melody #2?

Where presumably "equal" means the same melody.

I would say yes.

The method will depend on how you want to compare exactness, but I don't see how you can get away from comparing two sequences of values. If you encode both pitch and rhythm it seems you will in effect also have a notation system of sorts.

A pitch sequence could be something like: 0 +7 0 +2 0 -2 0 -2 0 -1 0 -2 0 -2 0 for the opening pitch changes in half steps to Twinkle, Twinkle Little Star. And 0 +4 +3 -8 +1 +2 -2 for the opening of Mozart's K. 545. Those would be encodings of pitch contour regardless of rhythm or key. Just step through the changes of each tune. If at any step the numbers aren't the same, the tunes are different.

You could encode differently and compare differently, but in principle it doesn't matter that much. The principle question is can two melodies be equated. Yes, that is possible.

1

If you want to see the waveform of a sound recording on your computer, download Audacity (it's free), load up the sound file and keep zooming in until you see the waves.

You will soon realise that the wiggly line tells you very little about the melody in the music. It's just too detailed.

In theory, there are a few ways to turn that wiggly line into an equation. None of them is remotely practical, or useful. By which I mean that if you tried to write down the equation (e.g. in a word processor), then the equation would be bigger than the original sound file!

An N-order polynomial would do it. If your sound is a 3 minute song recorded at 48k samples/sec, then the file contains 8,640,000 samples. An order-8,639,999 polynomial could exactly fit it. But that's a ridiculous thing to try to calculate.

Alternatively, do a Fourier analysis of the wave. Feed your sound file into a Fast Fourier Transform, and it will eventually give you back a large number of complex numbers, which define the waveform. The problem is that you've now turned a very large number of real numbers (the original waveform) into an equally large number of complex numbers.


If you are trying to reconstruct the music score from a recording, then it may be possible if it's a very simple recording. If it's one instrument, then that makes it a lot easier. If it's a whole band plus a singer, then you have little hope of getting anywhere.

But that's whole area of research.

1
  • I've actually seen users on the NinSheetMusic forums say they used the waveforms of the original video game music to determine particularly tricky notes. This implies that the wiggly line actually can tell you a substantial amount about the melody if you interpret it correctly. – Dekkadeci Nov 17 '20 at 12:56
1

If you want to describe a piece of piano music you could consider it as a function of one variable (time) with values in an 88-dimensional vector space. The vector components might be real (if you want to include loudness information) or binary (if you only want on/off information). I would suggest you begin by looking at MIDI files.

An alternative approach for melody only would be a 2-valued function of time, where one represents pitch and the other volume, or perhaps even omit the volume component. The graph of the pitch component will be a step function in Western music. The time variable can be either real or integer valued.

If you want to describe a sound as a mathematical curve Simon has described possible approaches.

Not the answer you're looking for? Browse other questions tagged or ask your own question.