An example of Flow Machines artificial intelligence music generator can be found on Soundcloud.

Has it been discussed or published to what extent their algorithms were developed with standard music theory concepts "built-in" as opposed to just letting the AI learn "the rules" of popular (or other kinds of music) completely on its own from examples?

There is a discussion here which I have read but do not really understand. Markov models are a general mathematical tool and not specifically related to music or human experience, and that's fine. But here I'm just trying to ask if music theory was included a priori or not.

Related: Algorithms for music composition but I didn't see Flow Machines mentioned here.

  • 1
    I've only skimmed the page, but the mention of "Machine Learning" suggests to me that they just throw a large training set of songs in the desired style at the algorithm and then let it loose. There are links to some papers on the page, though, which probably answer the question, I just didn't have time to read them. A quick skim of the abstract and introduction of the first linked paper seems to support my guess: the only concepts that are "built-in" are "two-voice melody", "harmony", and "meter", but the rules are all learned using Deep Learning. Mar 25, 2017 at 17:58
  • @JörgWMittag Thank you for your assessment. I just discovered that I didn't link "Discussion" correctly - I've fixed it now - it links to this overview which does go into more depth than the incorrectly linked page. Since I focused on the Overview page, I missed those open access papers you mentioned. I'll take a look at those this weekend. I'm glad you noticed and pointed them out, thanks!
    – uhoh
    Mar 25, 2017 at 21:13

1 Answer 1


I mean it uses a much logic as a Markov chain typically does which is use patterns in inputs to make an output. It will generate melodies, harmonies, rhythms, ect that are not necessarily music theory concepts. If you fed a Markov chain any book it will seem to make logical sentences, but it will not always be that way and the chain doesn't actually understand the language it just looks for patterns and the result may or may not make sense. This same idea goes for music as it will There statement about how it's done is:

The Flow Machines project takes a computer science perspective on style: how can a machine understand style and turn it into a computational object? An object that users can manipulate to create new objects with their own constraints. Technically we develop new technologies based on Markov models. Markov processes are well-known tools used to model statistical properties of temporal sequences.

In general the Markov chain alone is pretty weak in making music and in fact you will need at least a second order Markov chain to make anything make sense from a harmony perspective since tonal music's harmony goes somewhere. The key also is pretty important as the notes used in the melody and harmony will not be equally weighted for example how a C major chord is used in C major vs F major is different. In fact, the example video shows this weakness pretty well as the chords used and the melody kind of make sense, but it isn't very natural and has the meandering effect a low order Markov chain as the melody doesn't find a good point of rest.

They mention grouping styles which will help minimize these issues as mentioned in your second link:

We are currently investigating style and creativity under several perspectives: Technically we have made substantial progress in developing efficient Markov Constraints algorithms that can apply many types of constraints to arbitrary Markov models. Conceptually we are starting to build authoring tools in musical composition and text writing that enable people to generate content by manipulating the style of an existing author, possibly themselves.

Again this is not theory, but just using inputs to make an output. The chain won't understand any of the actual music theory concepts nor will it always do things that make musical sense. There's even a big red flag further down that page:

If you don’t like the proposed sequence the system will suggest another one, and so on until you are satisfied with the result.

Which is just a way of saying "take another chance and you may like what you hear". While using a Markov chain is better than just randomly picking notes, duration, ect, it's still a long way from being aware of what it making or any of the theory behind it.

I'm not going to speculate on the exact mechanics and without looking under the hood I won't know for sure, but based on everything and what they are describing there is little to no actual music theory behind it. The only thing that is kind of theoretical is grouping the pieces by "style" before making the chains, but that's not really music theory as it is just grouping pieces that whoever made the groups thought they were similar.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.