After some experimentation, I think I can give the answer myself:
What limited my picking speed so far, was a slightly un-ergonomic pick angle.
I chose this angle at slow speeds, because it gives me a sound, that is particularly pleasant to me,
but at higher speeds it becomes increasingly hard to hold this angle.
Stubborn as I am, I then tensed up ad higher speed, trying to force the pick through the strings the same way.
This is why I play so much faster after the gym:
Usually I train to failure, meaning, In my last set, I will do as many reps as it takes to totally exhaust my muscles.
For guitar purpose, I even recommend recovering for some minutes and then add at least one more set to failure.
Right after such a workout session, my muscles are substantially weakened. Sometimes to the point, where holding a bottle of water becomes a problem.
This is the state I practiced guitar in.
Now, instead of upholding the un-ergonomic pick angle, my muscles simply give way to a slightly faster pick angle, without me noticing it.
This is why I was faster, although, I didn't think I was doing anything different.
I repeated the experiment and can now reproduce the correct picking angle and higher speed, even without going to the gym first.
I have extensively experimented with different pick-angles before, but it seems, I never found the right one.
Partly because, I would start my experiments at very slow speeds.
At very slow speeds, my new fast angle sounds a little bit weak and I wouldn't have considered developing this "weak" angle.
If you want to try this experiment yourself, I recommend working your hole upper body, but especially your forearms.
A lot of back exercises like chin ups (depending on your grip) and rowing can be used for this.
If you have any picking issues, I highly recommend you try this.
It has helped me make a huge progress in a very short time, after years of stagnation.