Take the 2-minute tour ×
Music: Practice & Theory Stack Exchange is a question and answer site for musicians, students, and enthusiasts. It's 100% free, no registration required.

Graphical sound visualization (as available in several music players, eg winamp) is mostly used for fun, to add some visuals to the music, related to it in some way ( pseudo spectrum analysers, frequency vu meters, oscilloscopes, spectrograms, etc). But they do not provide a musically meaningful representation, they do not give enough cues -say- to "see" a major chord as different from a minor chord. I'm interested in this kind of sound visualization, such that the colours/shapes attempts to mirror/reveal the basic musical properties. Some desireable features:

  • It should have pure audio as input, not notes (not midi)

  • Uniform pitch change (chromatic transposition) should result in esentially equivalent visuals (apart from some trivial global change: translation, rotation, zoom...)

  • Intervals and chords types should be (up to a point, perhaps with some training) visually recognizable.

  • It could exploit sound dynamics (i.e., each visual frame not necessarily depends only on the current audio frame), but should be lightweight enough to be implemented in real time.

Does anybody know about some work (academic, commercial, whatever) along this lines? I've seen this, but it does not seem to lead very far...

(I have not the reputation to create an appropiate tag, perhaps "synesthesia"?)

share|improve this question
1  
Any decent fourier spectrogramm offers pretty much everything you asked for, when set to large enough window size (and thus slow response). –  leftaroundabout Dec 27 '11 at 19:01
1  
I'm voting to close this question as off-topic because it's not related to music practice or theory. –  Todd Wilcox Jul 29 at 21:28
1  
Bit confusing to learn that sound waves, their combination to form chords and the modelling of their physical characterstics have nothing to do with music practice or theory. –  user1019696 Jul 29 at 22:20

3 Answers 3

The problem with this is that distinct pitches have overtones that vary tremendously between different musical instruments. It's possible to analyze audio and reveal frequency information, but breaking that down into pitches and chords is something that technology is only beginning to grapple with.

That said, the Celemony company is doing amazing things with their Melodyne series of products. You must check them out.

share|improve this answer

Take a look at this demo. (Use 'View Page Source' in your browser to look at the code).

http://saebekassebil.github.io/teoria/

As a javascript library intended for use in the browser, there would be a bit of work to do to couple it up with pure audio, but I'm sure there are APIs and libraries around that would help.

It is certainly fast enough for real time applications :-)

It's open source. See https://github.com/saebekassebil/teoria

share|improve this answer

Display a spectrogram using windowed (short-time) Fourier transform, as seen here, for instance.

This methods divides the signal into multiple windows of equal width, covering a few seconds or about. They may overlap in some degree. Then spectrum is computed for each window, encoding the obtained values in brightness or color (maybe using log scale), and this makes one vertical column of the output image raster. The output has a vertical span of the selected frequency range and can span for many hours horizontally, as wide as the music sounds, because a well written program visualizing in real time should only use almost constant (and not increasing) amount of resources over time.

The obtained image shows very nicely notes, pitch bends, overtones and chords as they appear over time. This works well and is probably a standard approach. However if the instrument has very complex or unusual timbre, this may be seen as complex chords and be difficult to translate back into music notation.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.