Both C# minor and E major keys have the same key signature, so there is no difference there. This relationship is called 'relative key'. Each major key has a relative minor one, with the same key signature (to find it, descend a minor 3rd or ascend a major 6th from your tonic). Similarly for the minor key.
To sum up the difference: These two keys have the same key signature but different tonics.
The E major key would revolve around the note E, whereas the C# minor key would revolve around C#.
Does it matter which reference is used?
Of course it matters. A minor key sounds different than a major one, no matter what notes are included. So, you need to revolve around the minor harmony and not the major.
The relative key is not to be confused with the parallel key. The latter is when two scales have the same tonic, but one is a major one and the other is a minor one.
How do you tell which should be used?
Υοu see the harmony and the melody. In a song that is in C# minor, you are bound to see B# (leading tone) leading to C# (melody line). On the chords (harmony line) of the song you'll see V-i relationships, like G#-C#m etc. Things that will establish that the song is in C#minor.
Moreover, I'd like to add something concerning your enharmonic reference. Enharmonic note (or key signature)
is a note, interval, or key signature that is equivalent to some other note, interval, or key signature but "spelled", or named differently
A simple example: This example would be really easy to understand on a guitar or piano. C and D are a tone apart. If you ascend a semitone from C, you get C#; if you descend a semitone from D, you get Db. But both ways got you to the same note, each time spelled different. These two notes (C# and Db) are enharmonic notes.