Em G D A is
D major. I say superficially, because those chords merely match the key signature for
D major, but they would also match
A mixolydian, etc. etc.
If you had
Em G D A and at least
F# major or
F#7 for the dominant, and for clarity of a minor tonic, the
Bm chord, then
B minor as a key would start making sense. Making a case for a particular key when neither the dominant nor tonic chord is present is pretty weak.
Any of those possibilities of key or mode are only really made clear by defining the tonic, either the tonic scale degree or the tonic chord.
You can define the tonic different ways. The traditional way is harmonically by moving a dominant chord by descending fifth root progression to a major or minor triad. That second major or minor triad then becoming the tonic.
You can also define the tonic with melody. For a tonic of
D the melody would come to a rest on the tone
D or possibly the other tones of the tonic chord
You could also define the tonic with just chords and rhythm. Again, the basic idea would be to give emphasis to the tonic chord. You could do that by hold the tonic chord longer than the others. With chords
Em G D A you might do
||: Em Em G G | D D D A :|| (one chord per beat in that example) to rhythmically give more time to
D and make it the focus, the tonic.
You should be aware that your progression
Em G D A doesn't have a root progression by descending fifth, so the traditional way to define a tonic through harmony isn't there. In Roman numerals you have
D: ii IV I V. Assuming that repeats you have
V going to
ii. Traditional harmony would use things like
D: ii V I or
D: IV V I, something with
V going to
You don't have to make your chords progress that way. But if you are groping about for a tonic, you should at least be aware of what is happening harmonically in terms of dominant to tonic.