Can someone explain to me in layman's terms why that is the case?
If you look at the definition of a chord closely, you will see why they are the same. Let's look at the definition given by Wikipedia:
A chord, in music, is any harmonic set of three or more notes that is heard as if sounding simultaneously.
The key word there is set. A set in a mathematical sense is a special type of group where order doesn't matter and any of the elements(in our case notes) can be repeated and yield the same set. Because of this the notes
C E G,
E G C, and
C E G C all make a C major triad. The difference between each is just the voicing and the inversion of the chords.
For example a C chord with C as the lowest note is considered to be a root position chord, a C chord with E as the lowest note is considered to be in first inversion, and a C chord with G as the lowest note is considered to be in second inversion.
When the order is different the voicing of the chord is different, but the name of the chord is still the same.
I can give you the short answer: The chords are based on scale degrees. If you look at the C Major scale you get: C D E F G A B C
The major chord takes the root, the major third, and the fifth, or the 1st, 3rd, and 5th notes from the above scale: C E G
Once you have the chord, you can make inversions. This is helpful to change the sound of it slightly, putting different notes in the bass, and helpful on guitar where playing 1-3-5 isn't as easy with certain fingerings.
So the G-C-E chord you mentioned, is the same notes as C Major C-E-G, in a different order, or 5-1-3 which is the second inversion. The first inversion is 3-5-1 or E-G-C.
Edit: For comparison's sake, you might think at first glance that G-C-E is a G chord, but in G Major, C is the 4th and E is the 6th, so that would be 1-4-6 chord which isn't the major chord. G Major (1-3-5) is G-B-D.
You're right that changing the order of notes affects the chord. No matter how you arrange the notes C, E and G in a chord, it is still a C major chord, but there are other characteristics of the chord that are affected, and it will have a somewhat different sound.
First, a chord can be arranged in different inversions. You can spell a a C major triad C E G with the C at the bottom. This is called root position. E G C is a triad in first inversion, and second inversion is G C E. Each inversion has a slightly different sound and a slightly different function depending on the context it appears in. For example, root position is typically (but not always) somewhat more consonant than the other inversions.
One important use for inversions is voice leading. Depending on which instrument is playing the chords and the stylistic effect you're trying to achieve, you probably want to keep the chords from jumping around too much up and down the scale. For example, if you play a C major triad followed by a G major triad, both in root position, the range between the lowest note you play - the C - and the highest - the D in the G major triad - is over an octave. On the other hand, if you play the G major triad in first inversion with the B a semitone lower than the C, the range between the highest and lowest notes is only a minor sixth.
In addition, triads can be closed and open. The triad spelled C E G is a closed triad, because all of the notes are within an octave of each other. The triad spelled C G E, however, is over an octave wide and its an open triad. In general, if you're playing triads in lower octaves, like having a male choir sing the harmony, open triads sound better. Closed triads sound better in higher registers. This is because when notes are close together in lower octaves, they tend to sound muddy, but close notes in higher registers don't have that problem.
In conclusion, although the chords C E G and G C E are both C major chords, you're right that they're not identical. One is in root position, and one is a second inversion, giving them significantly different sounds and uses.
Here is a different take on the theme: let's find a nice common divisor of the given frequencies for pure intervals.
C E G -> 1:1 5:4 3:2 -> common divisor is 1:4 corresponding to C 2 octaves lower
G C E -> 1:1 4:3 5:3 -> common divisor is 1:3 corresponding to C 1½ octaves lower
E G C -> 1:1 6:5 8:5 -> common divisor is 1:5 corresponding to C 2⅓ octaves lower
So if you look for the frequency for which all of the given chord notes are an integral multiple of the frequency, you end up with C either way.
Because they sound similar!
C4-E4-G4 sounds similar to E4-G4-C5 because when you press C4 you get a lot of different frequencies (harmonics), most notably a large amount of C5 and some C6 too. Therefore inverting the chord doesn't alter the total sound very much. The sound of C5 was already there when you pressed C4.
(disclaimer: I'm a 100% layman, so this short explanation is as layman as it gets :-))
As others have mentioned, there are various different inversions for the chord C major: C E G, E G C, G C E. They are the same chord because they contain the same notes.
With this set of notes it turns out that C is the best note to pick as the root note. We then have a G which is separated from it by a perfect 5th (C-G,7 semitones.) The perfect 5th (frequency ratio 3:2) is the most important harmonic interval after the octave (frequency ratio 2:1). It sounds very consonant.
We could try considering E G C to be a modified version of E minor. E minor (E G B) has the perfect 5th E-B. So we could consider E G C to be E minor with an augmented 5th, but I can't think of any circumstance where you would want to do that, given that the (normal) name C major identifies a perfect 5th. Proper augmented chords like E G# C are dissonant, which is very unlike what we want to express.
Unless we have a very special circumstance, a chord which has a perfect 5th in it will have the two notes forming that perfect 5th named as the root and 5th of that chord. If it doesn't have a perfect 5th, it's already an oddball chord.
Sometimes there is more than one best name for a chord, and the name used depends on its function (i.e. how it fits in between the chords that come before and after) which is more important than which note it happens to have in the bass. See this question:Are there any enharmonic chords?
It is essentially an inversion of the C Major chord. In a major chord, there's the root (C), above that a major third (E), and then a minor third on top of that. (G) An inversion simply switches the order around, but you're still hearing the same notes. The first inversion has the root an octave higher than the rest of the chord. (E-G-C) (Forgive me if I'm using incorrect terminology.) The second inversion has both the root and the second note an octave higher. (G-C-E) If you continue this pattern of moving the bottom note an octave higher, you will have completed the chord inversion and will be back in triad form.