Why are some scale degrees avoided? What is the reasoning behind the avoid notes? Are there avoid notes in all modes? If so, which are the avoid notes of the different modes? Is this pattern widely spread through Jazz?
An avoid note is a chord tension which creates an interval of a minor ninth (or less) above a chord voice. Chord voices are the 1, 3, 5, and 7. Chord tensions are the 9, 11, and 13.
As I recall, the Imaj11 sounds terrible. On the other hand, the iimin11 is fine (because of the b3). Ditto the vimin11. And the IVmaj#11 sounds fine as well.
The avoid notes are fine in passing and arpeggios, but they are not necessarily agreeable in harmony. I have charted them here. The avoid notes are indicated in red; the affected chords are highlighted in yellow.
Here are some examples:
I believe that the best definition of the concept avoid note is the following:
Avoid notes: The pitch or pitches of a chord scale which are not used harmonically because they will destabilize the sound of the chord.
(from The Chord Scale Theory and Jazz Harmony by Barrie Nettles and Richard Graf)
This definition avoids the problems inherent in the other popular definition:
Avoid notes are notes that are not part of the chord and that are a half step above a chord tone.
(My own formulation of what I've read and heard many times)
In current usage this second definition is generally not correct, because it would define the b9 as an avoid note in all circumstances, which is not the case. In jazz, the b9 is frequently used harmonically in a dominant seventh chord which resolves down a perfect fifth. So in that context it is definitely no avoid note.
Another example of a note that would generally be an avoid note according to the second definition is the 4 in a major chord. In many contexts this would actually be a correct classification. However, especially on the guitar, there is a voicing which is quite frequently used in pop music. It is a standard C chord shape shifted up by two frets, resulting in the following chord:
D F# G(!) D E
In all contexts in which I've found that chord it was used as a chord with root D, so it's a D major chord (no fifth) with an added 9 (E) and an added 4 (G). Owing to the specific voicing and the timbre of the open G string, this chord sounds good in certain contexts. This is an example where the 4 is used harmonically in a major chord, showing that it cannot generally be classified as an avoid note. This answer discusses that chord in more detail.
Also check out this blog post for a more detailed discussion of avoid notes.
I'm sure more comprehensive and insightful answers will come, but it is common knowledge in jazz I would say. The reasoning is that the avoid tones clash with come of the commonly played chord tones. One example is the fourth of the major scale, when used over the I chord, since it clashes with the third (it's a minor 2nd between them). For C the third of the chord is E, while the fourth of the major scale is an F. It also creates a tritone with the 7th B. According to a survey mentioned in the Lydian Chromatic Concept of Tonal Organization, most people find the Lydian scale the most pleasing over a maj7 chord. Since the 4th is raised, the avoid tone is avoided in the Lydian scale. (It does create a minor 2nd with the 5th though...).
Avoid tones doesn't mean "never play", but rather "use sparingly or as passing tones". The patterns emerged since people disliked the sound of these notes together with the accompaniment.