"For each one of these chords, you need to choose a scale/mode/arpeggio, think of a melodic idea, and express it within the "shape" that corresponds to this scale."
I hate to be the bearer of bad news but this has never been true of any type of improvisation. I think you have a common misconception that Jazz is an analytical pursuit. One in which you use strict theory to reduce your options to only one or two allowed actions.
"Even harder, you need to choose guide tones for each one of these chords and make sure that your phrase will reach this target note in a smooth way, precisely on time. Of course, while all of that is happening, you need to choose rhythmic ideas, dynamics, articulation, etc."
What does that even mean? Why do you have to "smoothly" reach a "target note" "on time"? Any note can serve as an extension to the chord you are moving to so why not just let the note you land on BE that note?
"So I have spent countless hours memorising scales and arpeggios, and I played the right arpeggio up and down on each chord, and I tried to reach target tones,etc. But it is all so effortful that I always end up sounding like I'm playing an exercise."
Well I am truly sorry that in 20 years you spent your time this way. I would flip this over on you and ask you the following.
Have you spent countless hours LISTENING to bop?
Have you spent countless hours TRANSCRIBING your favorite bop solo?
Have you spent countless hours WRITING your own set of licks?
These are the three keys to understanding any musical style, Listen, Transcribe, and Write.
If you actually read through some of the transcriptions of great Jazz and Bop players you might find that more often than not they are not following these alleged rules you have listed. You are describing one of several approaches to improv that focuses on note movement within a progression and calls on classical multi voice homophonic harmony. It is not that the features you described are NOT useful or important but they are not necessarily used as a formula for developing ideas. By the same token, if you look at the sheet music for a lot of standards you will also find that your "rules" are not strictly adhered to. We use music theory as a guide and often describe certain things that work, when they do, but the doesn't mean we only do those things and nothing else. We seldom apply a template like this to build musical ideas from scratch.
There is one way to realize a flaw in the logic. The fact that you are "following chords" means you are giving precedence to the chord as the foundation of music, as the primary element. The fact is that the chord progression represents a secondary element to music from a classical point of view, and probably also from a historic point of view. The elements of Western music are Rhythm, Melody, and Harmony. Chords represent the third part, a supporting role. One can have meaningful ideas with only rhythm, and only rhythm and melody. Rarely if ever do you see Harmony without the other elements. What's the point? What I teach my students, and it was taught to me, is that we don't "follow chords", we lead them. They follow us (to be fair it's more of a mutual understanding in which we circle each other, like Bagua Zhang, but you need to understand the natural cycles in music to appreciate it). In reality there isn't much to a chord progression. The chords don't usually just randomly meander about all over the place but follow a pattern and that pattern is found in the circle progression and the circle of fourths.
I --> IV --> vii --> iii --> vi --> ii --> V --> I
Every bit of a Bop or Jazz tune can be pulled from here whether it's the main progression or just a cycle extension, even a key modulation. In addition to this circle are common chord substitutions and concepts from functional harmony. There is a natural movement within the group of chords {I, IV, V} that is embedded in the circle progression. Via application of the substitutes one can write this as follows.
I --> IV --> V --> I --> I --> IV --> V --> I
Much simpler. And easier to put ideas over. The fact is that one rarely needs or even wants to target key tones for EVERY chord in the progression but rather for those chords that occur at the end of a phrase, which might be 4 or 8 bars. Thus your musical idea could "conflict" with the chords in the progression but line up at the end and this would produce the feeling of resolution you want. In fact it is a more natural approach. We don't treat every chord in isolation but groups of chords as a single unit of "change". So one doesn't pick a different mode over the ii, the V, and the I but treats the {ii, V, I} and a single musical idea and builds up a phrase to play on it. Again, that phrase does not have to match chord tones on strong notes for each and every chord but at least have a feeling of resolution as we come to the end.
Another way to realize the error in thinking is that for a given melody one could have dozens of different chord progressions supporting it. What makes the one you see on the paper special? It might be crap! Rarely are lead sheets consistent with original scores and most are loaded with unneeded cycle extensions. Another thing I teach my students is to (1) write their own progressions for a melody and (2) learn to reduce those in the Real Book to the bare essentials by getting rid of cycle extensions (I call it cycle contraction). What you are left with may not be as interesting but it works and it's easier to follow. Then the student can play on easier harmonic terrain for a while and start adding complexity to a simple progression. In the process of doing this you start to hear the cycle extensions naturally and are inclined to include them even when they are not there.
Once you realize that most of these progressions are all the same and think that the formula works, the more interesting question is "How does anyone ever get an original solo idea?" Again, I would suggest that melody comes first. Some will criticize this (and have in the past) by saying that sometimes people come up with a riff first then add a melody line. I am not trying to pigeon hole the creative process but stating what is more often the case based on my experience. It turns out that there is a natural "movement" of melodic ideas in the Western music and the common chord progressions, like the I-IV-V and the circle represent that same common movement. So it should be possible to capture that.
Here are some pointers for getting past the barrier you seem to have encountered.
Practice voice leading over simple, very simple, two chord vamps. Try just "walking" over the I --> V7 --> I for several minutes. Starting by capturing one note per chord, then two, then more. Try FORCING a chord tone only on the beat at first to get the limited number of options in your inner ear then let yourself walk more freely "out" of the chord but always trying to bring yourself back. There is a very nice series of books called Complete Rhythm Changes for Guitar by Frank Vignola that takes you through a set of written solos. They start from the simplest 1-2 note per chord ideas then move into more complex lines. IMO, it is better to do this on your own, by ear.
Learn Heads! This is what the great bop masters, and creators, did. Don't just follow chords putting scales and arpeggios over them. Learn the melodic ideas of great songs. Improv is variation on a theme and it stands to reason that the best place to get ideas is the actual song itself. Commit the heads to memory and play them in all keys. Be as playful as you can by mixing and matching parts of songs. Since they all have similar progressions they are interchangeable. You will hear greats "quote" other heads and solos in their improv. This is part of the culture.
Listen, listen, listen. If you are practicing more patterns than actually listening to bop then there is something wrong. Listen to Parker and Diz, Pat Martino, etc. Listen to what they do. After a while you will start to realize that they reuse a lot of the same licks, just like blues and rock. Some are old cliches that go back to Benny Goodman or Duke Ellington, or even earlier, and some are "composed" licks that are signatures of the player. Point is that these guys are not "following chords" they are creating unique melodic flow that fits into the natural cycles of Western music.
One of the best pieces of advice for me came from a book by Jerry Coker called Pattern for Jazz. He talks about creating Licks and melodic lines without reference to a progression. Keep a journal of these licks. He recommends creating a few every day for the rest of your life. Get an idea that you really like or think is cool. Then take the basic pattern and rewrite in other modes, keys etc. If it is a Major scale idea rewrite it in minor or melodic minor. You can do this with melodic ideas from heads of famous tunes. Take a few licks from Ornithology and playing them in E minor instead of G major. See what happens! The last step is then to take your own unique ideas and try to fit them over chord changes. They will fit over some and not others. As you explore you will get it in your ear and muscle memory. You can also take YOUR IDEA and turn it into a diatonic sequence, thus allowing it to become a long melodic phrase that passes through several chords.
Take a song you really like and completely rewrite it. Learn by example what the structures of the song are by deconstructing it and rebuilding it. This will teach you a lot about music, the interplay between melody and chords and produce improv ideas. The problem IMO with the formulaic approach to chord scale matching is that it is too reductionist without the synthesis part coming after. Perhaps if you attend Jamey Abersold's Jazz camp one summer you will get both parts. But I suspect that, like many of us, you picked up a book or play along CD or attended a short master class and have not seen the big picture. If I'm wrong please let me know. As an example of this I might suggest All Of Me. This has a very distinct 3 note theme that is walked back and one can easily apply chord subs to rewrite the C Maj progression in A min (without transcribing the notes). Since C maj is harmonically compatible with A min, why even bother laying a C maj line over it! Just go to A min. Pat Martino does this a lot, he calls it "minorizing" the tune. Once you do this you find it more natural to default to fast repetitive minor Blues patters.
Keep trying and if something isn't working try something new. This is a long process that can get frustrating. The idea is to eventually relax into a groove, a flow, and ride the music. It can feel like playing catch up with the chords but with a better understanding of what the chords are supposed to do you will free yourself of the need to chase them with modes and just play melodic ideas.
Above all else, don't give up. I'd say if you do not have more bop on your iPod (or whatever) than anything else then get some. Listening is the key to understanding every style of music. To answer the question in the title directly, Nothing. Nothing is going through my mind when I solo.