Your song's chord progression is in essence I-V-bVII-IV (1-5-b7-4)
Looking at the roots by themselves, they are E, B, D and A.
the distance from E to B downward is a perfect 4th.
the distance from D to A downward is guess what? A perfect 4th too.
It is common in a song to first play 2 different chords, then reuse that same interval between the next 2 chords starting on a different root. (It doesn't necessarily have to be in that key's scale, as your D shows.)
Another song in E that comes to mind is George Harrison's "Got My Mind Set on You." It starts with C#m to G#m (its roots are a perfect 4th apart.) Then it goes from E to B (also a perfect 4th.)
You know, I've been busting my brain thinking I've heard this somewhere before. At first I wanted to keep going down this perfect 4th road, E to B, D to A, C to G. Then it hit me . . .
what if we take your example down to the key of D?
We then have D to A, C to G.
It sounds familiar . . .
Those chords are embedded in that prominent downward riff in Led Zeppelin's "Kashmir." Part of that is
G to D, F to C,
D to A, C to G ...) (*)
Thanks to you, we can now consider playing it in E.
(*) = the guitar plays these chords, the bass stays constant on D.