Music: Practice & Theory Stack Exchange is a question and answer site for musicians, students, and enthusiasts. It only takes a minute to sign up.
Sign up to join this community
What is a reasonably good way to detect chords in solo piano music?
Not the MIDI ones, but music generated by real piano/electronic keyboard.
I am trying to play a piano piece and check out its chords by feeding the piano output sound through microphone to my computer.
The best software that I've heard of is Melodyne. It's capabilities are truly amazing and I think it will do what you want very well. Take a look. It's not cheap though but I think it has a free trial!
There is software for that. But it is not always very accurate. Especially with multiple voices (or instruments) starting at the same time.
Research on this started over 40 years ago: Rabiner, Lawrence, et al. "A comparative performance study of several pitch detection algorithms." Acoustics, Speech and Signal Processing, IEEE Transactions on 24.5 (1976): 399-418.
And has improved a lot during the last few years: De La Cuadra, Patricio, Aaron Master, and Craig Sapp. "Efficient pitch detection techniques for interactive music." Proceedings of the 2001 International Computer Music Conference. 2001.
I suggest you take a look at the pitch detection plugin of the (free software) Audacity here.
There is an open-source project called CLAM.
It has an application that automatically analyzes chords:
http://clam-project.org/wiki/Chordata_tutorial
There are also more advanced applications that are part of the CLAM suite that can do more complex analysis.
Here is a demo of chordata:
Example: Yuja Wang Rachmaninoff rhapsody on a theme of Paganini 18th variation (rec. 2017) → to a fully automated generation of a solo piano reduction (MIDI &/or sheet music) in just five minutes
Huge advances have/are recently being made in regards to software that addresses tasks precisely like this one (and many others, such as image classification, "deep fake" video generation, natural language processing summarization, speech recognition, etc.), due principally to the arrival of new techniques I'm sure we've all (or mostly have) heard of -- that is, Deep Learning.
By far the best software for this I am aware of yet is called "AnthemScore". It's pretty incredible what it can do. Although it is 'trained' primarily on solo piano music and especially classical solo piano music (I would say check the site to ref the latest info/version updates though), nonetheless it performs rather astonishingly well on almost any song given as input. The output will at the least be very interesting and insightful in unique ways.
This program was trained literally for some 100 hours or so non-stop on some of the most powerful GPUs available on millions of input MIDI files as the training data: the site by the creator provides a much better and totally fascinating and generously transparent in-depth explanation of how the machine learning and thus ultimate magic of the software was created and functions - it has a provably (reproducible methods, etc.) demonstrated accuracy of about 90% for piano notes for solo piano music input of most all genres, but the accuracy is even higher for your run-of-the-mill classical music (say, Beethoven's Fur Elise which the site shows has near 100% accurately identified and auto-scored notes) [generally the more clear and organized the music, as all classical music pre-20th century generally is more so [than 20th century+], the better the SW output; but throw the latest most avant-garde pure electronic music into and the results are still absolutely interesting].
*Disclaimer: To be clear I am in no way whatsoever associated with this company, nor have any 'potential conflicts of interests' - in no way do I profit from purchases of this software. Frankly, I deeply wish it was open source and free and in my opinion, it should be but, the good news is it is definitely affordable and more than worth the price. *
For example, here is a youtube video I found (specifically listed under Creative Commons licensing [i.e. totally free for any use]) of Yuja Wang playing Rachmaninoff's famous 18th variation from his Op. 43 Variations:
and here is a link to the result (unedited) straight from AnthemScore (I downloaded the mp3 from the youtube video using the Linux tool youtube-dl), which I've posted in the public domain to my MuseScore account (according to the rights granted by the video being posted CC):
https://musescore.com/user/38320336/scores/7108932?share=copy_link
Is this score, purely aesthetically and visually, ideal, technically correct, and beautiful straight out of the raw result? No. (Ah, but aren't we already in Black Mirror creepy times, which ought to give us pause to ponder carefully: do we truly want THAT much automated work done for us? A perfectly readable entirely complete correct beautiful score spit out by the AI to us. Music editors would no longer be employable or valuable. Composers' skills would suffer from no longer ever being forced to actually write their scores/notated music. Still, I think as 💻-🪞 forewarns so too it glimmers - such advanced technology surely is inevitably well on its way for better or worse, yet on the other hand the ongoing democratization of music making would be only even further enormously amplified, and that's, of course, a good thing. I once heard John Adams say "[he did] not believe that [musical composition] fame is arbitrary" but that genuinely new, great ideas inevitably become well known. Without due time/room to develop what I'm getting at further, in short - I would summarize, that as with all things, probably it is a mix of both these two sides which would be best: knowing music notation ought likely to be a learned and well-practiced ability for any exceptionally innovative composer (timeless contributors - whose creations propel the never-ending evolution of humanity's musical artistic conversation to ever more sophisticated and rewarding and pleasurable, and insightful, direction. Hard to say! - We will have to wait and see until we find out for sure!).
Now that was in regards to the output of the score - the visual notation. In regards to the MIDI, however, does Anthem produce piano midi files that sound astoundingly sophisticated, approximating very closely what the very best human piano arrangement would sound like? Yes. This is where the SW best excels - at giving you the notes; the presentation of them as "readable" (i.e., elegant and minimally difficult to sight-read) sheet music is really an entirely different automation task of its own and besides any trained musician should not only be able to do their own score editing but if not prefer having it done precisely to their custom preferences then at least as a means of simply scrutinizing the music as closely as possible. I'd say, IMO, for this particular piece, the outcome is so beautiful and a total pleasure to listen to multiple times - and this is MIDI we must remember! - in ways, it has qualities surpassing the real performance. IMHO! There is for sure one "wrong" somewhere in there, for a smattering of milliseconds. And of course, it is unique - it is not an exact look-alike copy of Rachmaninoff's original piano reduction nor the much more recent fantastic Loveridge version (surely each of them spent hours if not days or even weeks preparing those scores by hand...).
I've experimented extensively with Anthem I would say overall, across the whole span of possible music (you can of course input any file whatsoever, it will produce results of some kind even for non-musical files if you want to get real...crazy? creative?) but for any even remotely harmonic tone based song (doesn't have to be rigid 12-tone Western traditional old music; could be anything, its capability for providing transcriptional results with high accuracy spans, so far as I have personally observed so far, the whole gamut of songs: a cappella polytonic chanting, Aeolian harps, Lady Gaga, Grimes, Elton John, John Cage, video game music/film scores - any youtube video..!) The accuracy of returned note identifications varies, unsurprisingly, mainly depending on how many layers/'tracks'/instruments the input song file contains as well as the complexity of drums and vocals present, those latter two which of course significantly complicate the challenge of creating an entirely automatic piano reduction - it must somehow cancel out the very very relatively most complex acoustics of those while both not entirely eliminating the influence they contribute to the song yet still being able to correctly parse out the base harmonics underneath it all.
This software performs excellently even on pure modern electronic music with zero traditional physical instruments whatsoever. It performs best of all on solo piano sound files, but you can feed it anything song and/or sound, and mathematically it will give you objectively close approximations to the harmonics present.
And further, quite curiously, in many novel ways, the SW picks up on aspects that it implements in the notation in ways a human never would have thought of and which we have never yet, which in turn can be very inspiring for recycling such new ways of transcribing back to a human-created arrangement and of course even entirely new original compositional ideas thereby can become apparent — if you look, and know how to look, closely enough for them.
And yes, MIDI is, of course, one of the possible options for export from the program.
MIDI, XML, .wav, or pdf, I believe are the filetype export options. I personally then import to MuseScore to do my by-hand score editing (if necessary; most of the time I am merely interested in gaining some insight into the music's compositional architecture (unfortunately I do not have perfect pitch, let alone on the level of being able to instantaneously with 90+% accuracy identify whole massive simultaneously sounded layers of notes - it would be much more difficult than being able to immediately play a Bach fugue by ear - and emphasis on immediately - maybe over several weeks spending half an hour a day I could finally by ear have transcribed a highly polyphonic Bach fugue, but life is too short frankly, lol, and why make music study/production more time-consuming and difficult than it already is, right? Again, we have very limited lifetimes, time in life, all things considered. It can not at all easily be argued that something which allows us — some tool (after all, as just one example of proof for this, dare to imagine the question of whether a pianist can perform without their piano [i.e., tool]? [Answer: That would be a definite nah.]) — to achieve acts of creation faster. This is precisely the "pro" of technology. Just re-iterating the obvious here! We have entered a new era of worldwide pan-cultural virtually exclusively-digital (at least partly; often entirely) music art creation. (Brave new world, yada yada yada.)
To reiterate, in the example above, I have made no by-hand edits* to the sheet music file produced by Anthem to illustrate what the software can do completely itself (i.e., exemplifying the raw output; put in a half hour of by-hand score editing and you've saved yourself many, many hours of work and could have something professional and satisfactory in quality).
*with the one exception the only thing I added was the piano dynamics mark (nonetheless all the notes come with specific velocity values directly inferred from the analysis of the input mp3 recording), and the pedal marks [I just ctrl-a selected everything and added one long pedal marking, simply out of haste]).
And finally, here is a screenshot showing what the analysis in AnthemScore for this example looked like (I did not perform any further manual edits in AnthemScore either; however, there is a whole suite of various built-in tools and features for doing so, in the latest "Professional" version):
Notice the note-detection being based on the interpretation of fundamental frequencies of the various harmonics present.
This is called transcription software; did you mean an app as in a mobile device app? Or desktop? There are many libraries that you could use if you were willing to develop your own app for either, of course, but there are also many already in existence. I'm not sure how accurate they are, but you can always give a couple a shot (try searching Github or SourceForge)! (Or just wait for someone else's answer)
Ableton Live has a feature built in called "Harmony-to-MIDI" where it analyzes an audio track (could be piano, guitar, whole song.. anything) and it outputs a MIDI sequence replicating the chords played in the piece. It even chooses the closest sounding synth patch to match the source. Also available is Melody-to-MIDI and Rhythm-to-MIDI. I'm not much of a keyboardist so I can't vouch for 100% accuracy, but it usually sounds pretty good to me.
It might not be worth buying the software for this alone, but if you are into recording and producing music Ableton is definitely worth a look. You can try to software free for 30 days I believe.
Edit: I just tried a program called Transcribe! and it is very useful for transcribing recorded material. You can load your mp3s and it will analyze the frequencies and tell you which notes are represented over time. Works for piano, guitar, voice.. anything with a pitch. It won't automatically transcribe all the notes, but it will plot out which notes are likely being played, then use your ears figure out which ones are correct.
Honestly, I'm going to have to give you the boring answer.
I'm sorry, but with all due respect to other responders, it's extremely difficult for any software program to detect chords accurately. For basic triads and progressions, some of the best software can do the job, but so much of chord naming depends on context. And isolating a fundamental frequency is no cakewalk, either.
For example, for simple note naming, how would a computer know that a note with the frequency of 440 Hz is a B♭♭ in a C diminished 7th chord? The best computers currently can do is just give the most common enharmonic spellings for notes, but that creates tons of confusion. Never mind the fact that even single notes can be extremely difficult to pick out of a song, and they tend to blend with their harmonics. Getting around that is possible if you can just hook up your note-naming software directly to the interface of a keyboard, though.
Some other concerns bother me as well. How is a computer to determine where one chord stops and ends? When I play a C major triad, then play the note B, is that Cmaj7 or just a passing tone? Maybe it's even part of the G major chord that follows? And how is the computer to rule out a B chord of some kind?
How can a computer handle the rootless voicings? Is that C♯ diminished 7th or A7♭9? Is E-F-A-B the basis of G13, or maybe an enharmonic respelling for D♭7♯9♭13, an altered chord?
The best way I can conceive of, until AI can just perform musical analysis faster than we can, is to learn a lot of music theory, build up your aural skills, and name them yourself, because no software on the market today can get the job done more accurately than a skilled human.
There's a reason Music's SE is so successful. We need humans to answer most of the good questions on this site, because context determines so much of musical analysis.
Others have suggested slowing down the recording. That's a novel idea, and whenever I transcribe from audio I make use of any way I can to slow down the notes. I cannot understate how effective that is.
A new app released today might be a sultion. http://audiokit.io/ I dont have iOS device to check the accuracy though. Will borrow one.
You can try Voice To MIDI Android application by BialaMusic BialaMusic, supporting live audio input from Piano and Guitar. Poliphony is supported as selectable option
Not sure of any other Sound2MIDI application
since sound is recorded to file first , than converted to MIDI, I am looking for live Sound2MIDI, Piano2MIDI transcriber, giving live MIDI stream as output
