There's a lot of difference between 'noodling' or 'widdling' and improvisation. One can use, say, a pentatonic and noodle over a three chord wonder all day long, playing long extemporisations without any mistakes being apparent. This can, however, be a great point to take off from.Only using, say, 4 of the notes, play a motif, perhaps 6 notes long. Over, for example, a 12 bar sequence initially. Keep as much of this as you can, while maybe missing a note out, or putting a 'bad' note on a weak part of a bar. Try keeping the note order, try changing the rhythm of those notes.Then try playing the sequence backwards, inside out, etc.With the chord changes, follow them with the motif, so they go up to a IV with the IV chord.Try not to make the improvisation too complex - imagine a non-muso is trying to follow or even hum your tune. Very complex will not necessarily equate to very clever.
That said, if you're trying to improv. over chord changes, as in a jazz situation, you'll need to start gently, with songs that don't go too wildly 'out of key'.Again, keep it straightforward and uncomplicated.You could follow the rhythm pattern of original notes, but use others instead. A set of notes (the scale) will often suffice, and initially, maybe, start each two bars with the root note of that particular chord. It makes it sound like you know where you are - only 'cos you do ! Either keep a rhythmic or a note motif coming back, every so often. Let's face it, if we play something that sounds good, we don't want to waste it, do we ?
I don't know how deeply you are already in following the changes, but hopefully this answer is going to be helpful.