I’m writing my first song. I came up with this chord progression and baseline. The progression is B A dm em played on the 5th and 7th frets. The bassline I came up with turned out to be part of the Dorian scale. Nothing I’ve been able to find this far has explained why the those chords work together. No key I’ve found can encompass all 4 chords. If any theory nerds could explain this to me it would be amazing.
The sequence as you gave it is...
|B A dm em |
...but if we permutate that to...
B|A dm em B|
...we can see that there are two tonic/dominant pairs shifted apart by a whole step...
dm: V i em: i V
Finding tonic dominant pairs is one way to try to identify a key, or at least typical harmonic patterns.
From this perspective, the only thing then needing an "explanation" is the shift from dm to em.
While those two tonalities do not fit into one diatonic set they do share 5 out of 7 pitches:
A C D E G. The Bb in D minor and the F# in E minor being the unique/exclusive pitches.
Those two keys are pretty close together, sharing many pitches. They are only two steps apart on the circle of fifths. It shouldn't be too awkward to shift between them. Notice that I use the word shift not modulation.
We can look at it a different way. Given that the Bb of D minor's key signature is not a pitch of the chord set while the F# from E minor's key signature is part of the chord set, and
em A B clearly fit into the key E minor, we could say these chords lean toward E minor with the
dm chord being the outsider. Maybe you could say it's borrowed from the phrygian where the F natural comes from E phrygian. But describing this as in E minor is probably wrong. The
A chord isn't really functioning like
IV in E minor.
This progression sort of reminds me of
V I IV bVII which is common in rock. If we give those chords letters...
B E A D
...it could be re-labeled...
E: V I D: V I
...we can then see tonic/dominant pairs separated by a whole step.
Your progression uses minor chords and so don't fit neatly into a single key with borrowed chord. But the chord root relationships are very similar to
V I IV bVII.
It is common for chord progressions to work neatly with keys and functional harmony, but that is not a law. There is no need to fit things into a key or a single diatonic set to analyze harmony.
If the key is A Dorian, then the A major would be similar to a Picardy 3rd (but I wouldn't call it that), and the B from A Lydian. If the key is E Dorian, the D minor is plain weird (vii?). If the key is D Dorian, then the B major chord is similar to a chromatic mediant.
I'm assuming the key is B, and the mode is actually Mixolydian. Playing a Dorian bass line under Mixolydian-based harmony is a common bluesy thing to do.
Ⅰ - ♭Ⅶ is very common in major keys, with ♭Ⅶ being the characteristic Mixolydian degree.
ⅳ - Ⅰ is again pretty common, normally found in progressions like Ⅰ - Ⅳ - ⅳ - Ⅰ. Such a ⅳ fulfills more a dominant, rather than subdominant role, because it shares two (leading) notes with the strongly dominant ⅶ°⁷ chord.
So, with one change, your progression would be one that could well be found in, say, a bluesy folk song:
Ⅰ - ♭Ⅶ - Ⅳ - ⅳ - Ⅰ
Here, ♭Ⅶ - Ⅳ is part of a typical fourth sequence, again very common in popular music, and Ⅳ - ⅳ is how the ⅳ pseudodominant is typically introduced.
So that's the light in which I'd see this progression, with the ♭ⅲ being the oddball. Work out what exactly the purpose of that chord is!
You are correct: one key cannot encompass all the notes you've used. But why should it? There are thousands of pieces out there that don't purely contain the notes from one key. There's no theory that says they should!
Often the majority of notes will belong to a specific key, and others appear, and do a good job spicing things up. If you're trying to find a key that all those notes fit from, no, it doesn't exist.
What happens in some pieces is what we call 'borrowing'. Usually from the parallel key. Not here, I don't think. There are 2 Ds - D nat. and D#; there are 2Fs - F nat. and F#. None of which fit into one key.
There probably is some theory which goes towards explaining why your sequence works, but suffice to say if you think it sounds good, be happy with that.
Bear in mind that all of the modes contain the same notes as each one's parent key, so looking for a mode here won't really be fruitful.
I don't think there's any accepted theory that would INSTRUCT you to write that progression. As @Tim says, we could possibly concoct one to describe it. But we could do the same for any random progression.