Like @Tim said. But let me expand a little.
Accept one basic concept. 'The key of a piece is a framework, not a restriction'. There is nothing wrong, unusual, or even worthy of more than passing comment when a song uses a non-diatonic chord. Then you'll not need complicated justifications like 'borrowing' (so OK, that chord's diatonic in some OTHER key? So what?) or 'modal interchanges'. You just need 'so what's that chord doing there?' And the best answer may well be 'because it sounds good.)
If it was F# (or F#7) we can easily hang a label on it. It's being a diatonic dominant (or dominant 7th). Good ol' hymnbook harmony. We could label F#m as 'borrowed' from the parallel minor. But does that help? (Anyway, it's quite likely a tune in B minor would still use the major dominant chord, then we'd have to use the same excuse backwards!)
Just accept that a diatonic chord with one note modified is a 'thing you can do without sounding particularly left field'. In a harmony exercise that F#m MIGHT signal a modulation to E major, via F#m7, B7, E. In that case there would be some point in considering it 'borrowed' from E major. But that's not, I think, what happens here. It's just nice.