You are asking for more than one thing here. The first is how to improve your finger style technique and the second is how to improvise finger style. As for the first the best way is to work on some classic right hand exercises to improve technique and make your right hand comfortable with different picking patterns. I do not know if there are any special ones in the folk, bluegrass etc styles but in classical guitar you have the Giuliani exercises, and Papararo exercises. You can google each and probably find a free pdf. In a nutshell these lay down various right hand picking patterns and apply them to 2 chord vamps, like C --> G7 and back. So in a sense you can make up your own versions. Similar, but more complex, exercises exist in Carcassi. I am sure you could find a Bluegrass finger style exercise book if you search. I use the Giuliani and Papararo with my students who want to learn this type of playing.
You second question is a little more along the lines of "How do I playing improvised patterns on the fly". And you give examples. To that I would say (1) I'm not sure these guys are improvising, they may work out these patterns and a lot of great players do, (2) practice over the years is what provides you with the tools to make changes on the fly. For example, you may be playing the same chord progression but throw in a new arpeggio or tremolo here and there. That is a very light form of improv. Moving farther from the plan requires a bit more knowledge of what you can do and a good ear. If you are interested in improvising chords to a song or adding additional chords to a song then I'd say you want to delve a little deeper into music theory.