I am not a great improviser (I don't practice that anymore), but back in the day when I was doing a lot of soloing on saxophone in jazz band, I had the same trouble when I started out. A few things helped me (some of which I think you are already doing):
1) listen to A LOT of music that you are trying to play. For me, that meant listening to a ton of jazz. This did not help immediately, but it filled my head not with specific ideas, but with the general feel of what to do.
2) Practice the technique and the riffs and LOTS of scales and arpeggios. These things are the basics of your instrument. My teacher told me "your instrument needs to be an extension of your body." When you think a note, your hands need to be able to go right there and make the sound come out. The only way to get there is to know your instrument like the back of your hand. Scales and arpeggios and more scales and arpeggios. Riffs are OK, but scales and arpeggios let you make up your own riff. If you want to practice riffs, transpose them to more keys.
3) This was the one that helped me finally "do it" and play a nice solo. Start simple. A lot of the reason I was having trouble was I was biting off more than I could chew. The songs had too many chord changes or went too fast. Finally I had a song to solo in that was a slow minor-blues I really connected with, and it all clicked. I suggested starting with blues. Really get comfortable with that before trying to solo with music with more complicated chord progressions and sets of scales. Give yourself time to soak it in, relax, and feel it.
Adding a 4th suggestion: sing along with the chord progressions. You can't play a solo if you can't hear a solo.