My experience of improvisation started out in jazz bands, although over the years I've adapted to other styles too (including some church bands).
The best technique I found in the early stages was learning a few scales which work well over common chord sequences (previously I was totally unaware that there were any scales besides major and minor!). The classic jazz/blues improvisation I started with was the pentatonic minor scale, over a 12 bar blues progression.
For example, if you're playing in C major, your basic 12 bar blues chords are C7, F7, and G. Play that with your left hand, or use a backing track. Meanwhile, with your right hand, play notes from the C pentatonic minor scale (C, Eb, F, G, Bb). Make sure you vary the rhythm and volume of the notes as you go along.
As you get more familiar with the scale, add in some more notes. In the above example, F# can act as a 'blue note', which works well when transitioning to F or G. You can also use Eb as a transition to E natural. As you get more comfortable, start including multiple notes in the right hand, based on the current chord. Later, start varying the chord progression too, throwing in some 9th and 13th chords instead of just 7th's. That lets you add D and A into your melody at appropriate times.
The goal is to get really familiar with the basic scale (whichever one you're using) plus the extra notes which can go along with it. Keep playing and improvising with it, over and over and over until you can rattle it off blindfolded. You might sound terrible at first, but stick with it. Eventually you won't even be thinking about the scale any more -- you'll just feel the music, and your fingers will do the work.
Obviously there are other musical styles out there besides jazz and blues, but the same principles for improvisation apply. You could perhaps buy a book or look at some websites which explain applications for the different scale types.