Let's assume I have gathered a large collection of data, such as 1000s of triangles, consisting of 3 points each, in a defined 2D space, consisting of 2 values for each (X,Y) and a color value for each triangle consisting of 3 or 4 values (R,G,B[,A]). Would it be possible to "transform" this info in "music" or at least in a harmonically consistent series of sounds? I understand some basic sound properties such as Frequency, Volume, Duration, Attack, Sustain, Tone, etc. Is there a suggested way to correspond values to characteristics-properties?

I understand that this would probably produce more of an ambient set of sounds than "music" but I am just looking for a way to "translate" any data in "pleasant" sound environment.

  • 1
    Hey, a genuine application for a neural network! (both funny and serious here) – Carl Witthoft Oct 26 '18 at 13:17
  • 1
    Brilliant question! How you go about determining the traslation into music probably depends a bit on the nature of the data: eg do lighter colours mean something rates highly (might make them a higher note or piercing tone) or does their order left-right determine a scoring? Or just completely random, eg peak points on mountains on a map ? so .. do you have any idea of the origin of the data, or is it literally any abstract data, the nature of whcih you don't know ? – user2808054 Oct 26 '18 at 14:12

The key to creating a musical piece that sounds halfway bearable is to constrain the parameters that you associate with values in the real world.

For instance, if you write in the traditional chromatic scale and assign pitch to represent any kind of input variable, the result will almost always be unbearable cacophony. Using a simple major scale will sound only slightly better. A pentatonic scale allows for fewer harsh dissonances, so its easier to arrive at something that is at least tolerable ambient sound.

Of course, getting any kind of reasonable musical macrostructure from external data is almost certainly hopeless. Human perception of music obeys largely unconscious but nevertheless specific rules. For instance, even if you constrain yourself to using only I, IV and V chords it's possible to write rather satisfying chord progressions. But the chance that an effectively random chord progression sounds anywhere near as satisfying as the one in even the simplest blues or rock song is negligibly small.

  • I think you can make a similar point about rhythm. I wrote a 'random' ABC music program a long time ago. I initially put all my attention into the pitch constraints. But when the program randomly selected rhythms from a palette of figures I learned quickly that rhythm needed to be constrained. – Michael Curtis Oct 26 '18 at 18:08

I am just looking for a way to "translate" any data in "pleasant" sound environment.

If your data is non-musical, The amount of work you need to do probably depends on how broad minded you are about what's 'pleasant'!

You could make quite a direct transformation - for example, you could decide that one dimension of your 2d space will map to time, and another will map to pitch - and you could generate a sine wave at the time and pitch that corresponds to the average value on each dimension on each of the triangle's points. You could say that the duration of the sine wave will be proportional to the length of the triangle's sides. That would make some kind of noise - run it through a reverb and it might even sound ambient - but maybe not to everyone's taste.

If you want something that sounds more traditionally 'musical', then you will need to do more of the work that a composer would need to do when writing any piece of music - you will need to make choice about what sounds will be playing in your piece, what kind of scales and harmonies you want to hear - in other words, the constraints that Kilian Foth talks about - and set up your data to select from those pre-ratified choices in interesting ways.

If you are already able to compose a piece that you find pleasant, then using data to drive choices within the constraints of that style can be interesting - but it's probably harder than just writing a piece without using the data!

  • Measures can always be combined e.g I thought i could use triangle area or color luminance as a volume measure, because these measures are easier perceived as big-small (intense-soft) and thus related to sound being big-small or lintense-soft – kokobill Oct 26 '18 at 8:35
  • @kokobill yes, sounds sensible. Mappings to volume or parameters of timbre will probably be less likely to need further transforms / constraints than mappings to time and pitch. – topo Reinstate Monica Oct 26 '18 at 8:42

An approach can be outlined as follows:

  1. Feature extraction: First, list a set of features from the geometry/data that you wish to represent musically.

  2. Mapping the features: Once that is done, find a mapping from those features into a set of musical features of some choice.

  3. Sequencing the mapped musical elements: then the mapped musical features may be sequenced in a musically aesthetic/pleasing manner.

So the content will be dependent on the content of the geometry/data, but the music will made by an algorithm from the content, to make it pleasing. One can have variations in this scheme to bring as many features in and at the same time to make the music as pleasing as possible in a specific genre.

For the specific case provided in the question, we can think of the following approach:

In this case, if the data consists of only triangles and let's say their vertices are colored with 3 colors, you have all in all 8 possibilities. which can be converted into say chords. Frequency of occurrence of specific triangle coloring can be translated into frequency of occurrence of chords. The set of chords each with a certain frequency of occurrence can finally be translated into a pleasing sequence by applying music theory such as chord progression.


X - Octave(Middle octave C4 to B4)

Y - Volume

R - Pitch/Note (C,Db ,D, Eb, E, F,... )

B - Note duration(Rhythm)

G - Dynamic ( f,ff,fff,p,pp,ppp)

A - Midi instrument

Try above items with different combinations, you can also try chords,minor scale, major scale, pentatonic, interval attributes as you develop the system.

Not the answer you're looking for? Browse other questions tagged or ask your own question.