TL;DR: I think the answer you are looking for is yes. If you want to improvise for a specific audience, then having a form, some kind of structure, your piece would not seem "random", because it's easy for people to follow forms, especially if they are familiar with them.
Long answer: What are you aiming for? Do you want to improvise an entire piece live? Do you want to improvise so as to come up with a melody or chord progression? Do you want to improvise something and then work on it and compose a piece off your improvisation? Do you want to improvise on top of a sonata form?
All the above are different. Bach used to improvise, yes. But he had spent all his life composing, studying and improvising on very specific forms. So, Bach could easily improvise a 3 voice fugue, something us mortal might take years of studying, just to compose one!
Jazzists have a form in mind (AABA / AABC / ABC / Whatever) and improvise on top of that. A lot of composers improvise on an instrument (most commonly on a piano) to come up with an idea and then use that idea as the foundation for a piece they'll work on.
Improvising on a certain form can be of advantage if you are going for a more structured piece. But it's not uncommon to improvise freely (free improvisation). No form, no pre-thought ideas, nothing. Just getting up on stage and improvising. Form is somewhat restricting, because you have to follow it, but it's easier for people to follow a AABA form rather than a ABCDEFG... "form".
Keep in mind that most people nowadays are familiar with the popular song form: Intro-Verse-Chorus-Verse-Chorus-Bridge(?)-Chorus(usually)-Outro. So if you improvise on a form of this kind (with slight variations or not), most people will be able to easily follow your piece without trying. But if you want to go beyond that (something that I'd highly suggest you at least try), people might find it more difficult to follow, but no one is stopping you from inventing a form of your own!
My suggestion would be to get familiar with the form you like. This way the forms will come natural to you; you won't have to think about the different sections of the piece, and you'll be able to improvise on them. The first time you'll try to improvise on a AABA form you'll mess up. The second time you'll mess up. The third time you'll do a little better. After you've practiced it a lot, you'll be able to do something solid a specific form.
It all boils down to having a structured way of thinking.