In F major, the bVII chord will be Eb which is normally said to come
from F minor (F aeolian). But my understanding is that this chord can
just as easily come from F dorian, or F mixolydian.
It's a matter of perspective. If the person in question looks at the Eb chord as a part of an F minor context with chords like Bbm, Ab, Db, etc. and uses that harmonic palette to draw from - with the intention of quickly returning to the original palette - then that's where it "comes from". And then it is reasonable and logical for the person to say, "I borrowed the Eb chord from the key of F minor".
However, if the person is thinking about, say, F mixolydian, then the right thing is to say "I borrowed the Eb chord from F mixolydian." If you feel like you're moving to the other mode more permanently, then you say you're modulating or changing the mode, not borrowing. Right? You say what you do, if you want others to know what you do if it isn't self evident. If you think you're riding a bike then you say "I'm riding a bike". If you think you borrowed the Eb chord from F minor, then you say "I borrowed the Eb chord from F minor." If you think you borrowed the Eb chord from F mixolydian, then you say "I borrowed the Eb chord from F mixolydian."
An Eb chord alone doesn't make the mode F minor or F mixolydian or anything. But F minor or F mixolydian or F dorian etc. might be on your mind, depending on what other things you do or intend to do.
If your question is, "why do so many people so often think about this one particular mode instead of all these other modes", then maybe it's something cultural. It's a popular thing to do for some reason.