I'm trying to look at the fret board in distance vs notes, I know that going up from the low E string to the A string is a perfect 4th, but going from the A string to the low E string is a perfect 5th. Why is this? I would think there is some symmetry between the strings (excepting the G string)
The question is asked with slightly wrong premises. Any interval is named from bottom to top - lower note being the start point. So E to A will always be P4 - count up - E F G A in letter names. count down in letter names, it's the same P4!
Start with A underneath, and it's P5. A B C D E, so it's never going to be P4. No need to worry about number of semitones for this particular case, honest!
You've also confused the issue - header is about A and D, whereas question is about E and A - although exactly the same applies. Try counting the other two.
With intervals, there's the 'rule of 9' It concerns inversions. A>E is P5, whereas E>A is P4. major intervals become minor, and vice versa, as in M3 inverts to m6. Augmented and diminished are likewise - C>F♯ is aug4, whereas F♯ >C is dim5.
It's all about the direction in which you're moving.
If you're moving from A up to E, it's a fifth, because A–B–C–D–E is five pitches. But if you're moving from A down to E, it's suddenly a fourth, because A–G–F–E is only four pitches.
This is because the perfect fifth doesn't actually split the octave into two equal halves. Since the perfect fifth is itself inherently asymmetrical, some intervals won't seem to match up if we're not careful about direction.
Similarly, moving from the D (up) to the A string is a fifth. If we want that same fifth from A to D, we have to move A down to D; otherwise, moving A up to D is only a fourth.
As we can see there are 2 symmetries in the 12 keys. But there will be augmented fourths and diminished fifths from g#/ab to d and d to g# or ab. And it shows clearly that the interval e to a is 5th (down) or 4th (up).
On the guitar you play efga on the E - string in the bends 0135 and abcde 02357 and you see there is fourth and a fifth.
Or you play the whole major scale doremifasolatido on one single string E or A: whole steps you have to skip a bend!
0-2-4-5-7-9-11-12 WWHWWWH w=whole steps h=half steps
We can see and hear that there us no symmetry but 4ths and 5ths.