I am writing a song with the verse going from C > Am > Em > Em. . I can't think of any other pop/rock songs that use this progression and if anyone can think of any please share as this would be very helpful right now. So I suppose the progression could be either E minor or E phrygian. If the vocal melody above these chords was to exclude the F/F# note would the listener ever feel like there was an implied tonality?
Even though the chords could be taken from E phrygian, it would be very uncommon to say that the given progression is in E phrygian. The reason is that the character note in phrygian is the b2, which would be F in this case, and since the note F does not show up anywhere, the most common interpretation would be to say that the progression is in E minor.
The story would be different if there were a melody over these chords which uses the note F. But if neither F nor F# are used in the melody, this progression is most likely in E minor.
Generally, a mode other than major and minor is only perceived if the chords and/or the melody feature the mode's character note. E.g., it doesn't make much musical sense to say that the progression C-G-C is in C lydian (with the #4, i.e. F#, as character note), even though the chord tones are all part of C lydian.
Seems more like C, although E Phrygian contains the same notes exactly. When it gets written on the stave, you need to decide what the key sig. is going to be, and to me no # or b fits better. At the end of the song, you'll find one particular chord will sound like it fits better than any other. That'll be the decider for the piece's key.