It is said that the Ionian (major), Aeolian (natural minor), and Locrian modes weren't really three of the ancient church (Gregorian) modes. What is the reason?
Historically, the modes were methods of classifying chants. There were few if any chants having the note patterns that fit the Ionian or Aeolian or Locrian modes. Later (1400s or so) Locrian wasn't used as the natural fifth above the final note (B) was diminished. Actually, in the earliest known chant descriptions, the note B was mutable (B soft written "B" being Bb and B hard written "H" being B natural; this practice has been followed in German writings.) (Other languages may have written them differently; some used B-moll and B-dur or the like.)
The modes roughly described the range of chant melodies. From D to d (an octave with scale steps D-E-F-G-A-G-B-C-D with half steps between E and F and between B and C) was the normal Dorian; lower would have been A-B-C-D-E-F-G-A (called Hypodorian meaning below Dorian) to describe chants that had a different range. The final note in both cases was a D. (The pitches were relative, not absolute.) This would have been the Aeolan were the chants to end on A but most ended on D in this classification. Similarly for Hypolydian (C-D-E-F-G-A-B-C) with F being final and Hypophrygian being B-C-D-E-F-G-A-B. The scale patterns existed but were not considered separate as the chants didn't end on these notes.
Hypomixolydian had the same pattern as Dorian but with different endings.
During this time, the hexachord classification was also developing and used along with the modes. The "soft hexachord" F-G-A-Bb-C-D, the "hard hexachord" G-A-B-C-D-E, and the "natural hexachord" C-D-E-F-G-A-B. These were written in an overlapping manner and allowed one to describe "modulation" between modes. It was very complicated. Check Google and the Wikis about chant modes and hexachords.
Later in the 1500s some authors did allow for 12 modes and some for 14. With the invention of F# and Eb then C#, etc., things got more complicated until by the 1800s, the current naming became common.
Ideally, we like to imagine 7 modes for the 7 tones of the diatonic scale:
- A = Aeolian
- B = Locrian
- C = Ionian
- D = Dorian
- E = Phrygian
- F = Lydian
- G = Mixolydian
In fact, motives can be found in all 7 tonalities (the other answer misleads on this point), but their classification was perceived in the first millennium as a system of not 7 modes but 4 (doubled to 8, for there were two versions of each mode, as described in the other answer):
- The First (Church modes I and II) = Dorian, Aeolian, or both
- The Second (Church modes III and IV) = Phrygian, Locrian, or both
- The Third (Church modes V and IV) = Lydian, Ionian, or both
- The Fourth (Church modes VII and VIII) = Mixolydian
What might be dubbed the Fifth, Sixth, and Seventh in a "pure" heptamodal system are seen rather as variants of the other four. This is because of the flattenable B, a tradition that appears to date back to, or at least has theoretical continuity with, the principle of the "diazeuxis" in the musical system of Ancient Greece.
The custom of permitting B as both natural and flat in the course of a diatonic melody introduces an allowance for "one degree of modulation" of the tonality, without actually deeming a change of mode. For example, if we allow for B-flat in the D mode, D takes on an Aeolian character, since its intervals are now identical to the A mode when the A mode has a B-natural. Thus:
- A mode (naturally Aeolian) + B-flat = A Phrygian (= E mode)
- B mode (naturally Locrian) + B-flat = B-flat Lydian (= F mode)
- C mode (naturally Ionian) + B-flat = C Mixolydian (= G mode)
- D mode (naturally Dorian) + B-flat = D Aeolian (= A mode)
- E mode (naturally Phrygian) + B-flat = E Locrian (= B mode)
- F mode (naturally Lydian) + B-flat = F Ionian (= C mode)
- G mode (naturally Mixolydian) + B-flat = G Dorian (= D mode)
Trim off the redundancy, and you get a minimum of four possible modes, with one degree of modulation:
- The First = Aeolian <--> Dorian
- The Second = Locrian <--> Phrygian
- The Third = Ionian <--> Lydian
- The Fourth = Mixolydian
De modis quatuor et affinitatibus vocum.
Cum autem septem sint voces ... septenas sufficit explicare, quae diversorum modorum et diversarum sunt qualitatum. ... Et nota, quod se per ordinem sequuntur, ut primus in A. secundus in B. tertius in C. Item primus in D. secundus in E. tertius in F. quartus in G. Itemque nota has vocum affinitates per diatessaron et diapente constructas: A. enim ad D, et B. ad E, et C. ad F, et D. ad G, a gravibus diatessaron, ab acutis vero diapente coniungitur hoc modo:
On the four modes and the affinities of voices:
While there are seven voices [tones], then, ... it suffices to explain the seven, which are of diverse modes and qualities. ... And note that they follow themselves in order, as the first on A, the second on B, the third on C: likewise the first on D, the second on E, the third on F, the fourth on G. And likewise note these affinities of voices, constructed through a diatessaron [perfect fourth] and diapente [perfect fifth]: for A is joined to D, and B to E, and C to F, and D to G, by a diatessaron from the lowers, or by a diapente from the uppers.
These "tonal affinities" are the underlying principle that makes the circle of fifths possible. If we were to hypothetically allow for not one but two degrees of modulation up or down the circle of fifths, i.e., two flats for example, we would have three modes covering all 7 diatonalities, for example:
- First = Aeolian <--> Dorian <--> Mixolydian
- Second = Locrian <--> Phrygian <--> Aeolian
- Third = Ionian <--> Lydian <--> ...
Or for three degrees only two modes:
- First = Locrian <--> Phrygian <--> Aeolian <--> Dorian
- Second = Mixolydian <--> Ionian <--> Phrygian
At six degrees, we have only one mode, whose melodies can be composed in all seven tonalities of the original modes.
If we undo all those allowances and forbid flats and sharps in all chant melodies, then we will have to classify 7 modes, one for each of the 7 tonalities, unless of course we forbid the Ionian, Aeolian, and Locrian for example. But that simply did not happen in Gregorian chant, where you can find the interval patterns of all seven modes. It's just that they are perceived as four.
This kind of perception is not dissimilar to today where a minor-key song can have Aeolian, Phrygian, and harmonic minor elements, but we still think of it as "minor" and often don't even notice those other elements until we analyze the piece.
The other answers are interesting and informative, but there is a simpler answer: those three modes were identified and named at a much later point in history. The Aeolian and Ionian modes were introduced to music theory in the middle of the sixteenth century, hardly "ancient" history, and the Locrian mode had to wait another couple of centuries, not being introduced before the eighteenth.
The other answers describe the reasons behind this timing.