I think that the standard Italian markings and terms (tempo marks (and the word "tempo" itself), dynamics, and articulations at the very least) are so ingrained into our musical language that we don't even think of them as Italian. In a sense, "legato" is the word for "smooth and connected" in the language of music, which is merely borrowed from Italian.
With that in mind, while I would agree that mixing languages is to be avoided, I think there's nothing wrong with marking your piece "allegro", indicating "cresc.", but then using "angrily" or some other uncommon style indication. Grainger is known for translating all of the common Italian markings to English ("louden" instead of "crescendo", "very fast" instead of "molto allegro", etc.), and the result is somewhat jarring to read. As an experienced musician, my brain actually translates these back into the Italian terms to interpret them.
From my experience, I believe this kind of mixture is common in new music, for increased specificity as you describe. And I think it's a failure of communication if you go through the effort to translate to Italian, and then the performer has to translate it back to know what you meant!