As you point out, of the seven modes of the major scale, the Ionian (major) and Aeolian (natural minor) are 'preferred', or perhaps used more often. Also, the labeling system seems to accord special paradigmatic value to these two modes.
Now one should note right of the bat that there is plenty of music (popular and otherwise) that uses melodies and chord sequences that are derived from the other modes, especially the Dorian, Mixolydian, and Lydian mode.
To come to your question: is there any structural difference between the Ionian and Aeolian mode, and the other five modes of the same system? Here is one structural difference that is often overlooked, which is related to the tonic or 'home chord' of the section of music that is composed in major or natural minor setting:
In the context of tertiary harmony, which means harmony in which the chords are formed by stacking thirds from the scale, we should expect each of the seven modes to come with a unique set of chords (major/minor, but also seventh chords and up) around a given tonic chord.
If we take C major as an example, the home chord is the C major chord (C, E, G). For the piece to be meaningfully "in C major", we have to use the notes of C major but also use the C major chord as a home chord.
If we now switch to the relative minor scale of C major, we get A natural minor. Again, this means that we have to use the notes of the A minor scale as well as the A minor chord (A, C, E) as a home chord.
For each of the five other modes, we get an analogous story. Now what is special about using A minor or C major as a home chord in this context? The short answer is that these two chords are the only two triads that exist within this system which contain both the notes C and E. The reason that this is of interest is that the C major scale (or any of its equivalent modes) contains exactly one tritone interval, between B and F. This interval is often used to generate tension. For that tension to resolve, both notes of the tritone need to move to adjacent pitches of the system, to a consonant interval. Using all the pitches of C major, the notes C and E are simply by far the most pleasant way to resolve the tritone between B and F.
This explains why A natural minor (with A minor as a home chord) and C major (with C major as a home chord) are special. These are the only modes/key centers which contain the interval (C-E) which is the optimal way to resolve from the tritone. So whenever tritone resolution is involved to go on musical journeys (tension-release), then the Ionian and Aeolian modes are good, because their tonic chords are those which feel resolved when coming from a tritone.
Incidentally, in the period that these labels were invented, tritone resolution within this type of system was the preferred approach to tension and release in music. So the labeling system reflects the fact that only these modes can be 'home', if 'home' means "a chord which has an interval to which the tritone resolves."
This is of course the beginning of the story. It also contains the beginnings of an explanation why the harmonic and melodic minor scales were invented. I hope this helps!
