You might have to change a bit the way you're strumming the strings.
As you're only playing on the bottom 3 strings, you should focus on your strumming hand in order to reduce your movement, maybe you go too far. This is not a "full chord" strumming, you should stop your movement sooner in order to avoid touching the 3 string you don't want to play.
Moreover you just have to train in order to release pressure on the other 3 strings in order to mute them with the same finger (the same exercise you need to do to play a proper A chord with one finger).
Combine this "light" muting and a reduced strumming movement and you should be able to play nice Drop D powerchords.