Generally, the neighboring keys in the circle of fifths are good candidates point for any chord borrowing. D-major is only one sharp away from e-minor, i.e. the scales are very similar.
Specifically, VII acts as a dominant in a natural minor scale, and IV is a dominant to that, i.e. a double-dominant. This is arguable in your example though since there's a VI between the IV and the VII.
Why going to neighbours in the circle of fifths works so smoothly can be explained easily enough on grounds of physics: a fifth is a frequency ratio of 3:2, which also occurs very early in the overtone series of most instrument sounds. So, adding a fifth on top of any note that's already in the scale will hardly get you out of the scale at all; in your example, the C♯ in the A-major IV chord is actually already present as the third harmonic of the scale note F♯.