What does "pull" mean?
Are you taking lessons?
You state that you are working as a programmer. I program as well and play guitar. To answer you questions directly:
No, you should not do "gymnastics" that may not help.
Yes, it is sometimes normal to experience discomfort when you start playing a musical instrument because you are using muscles in a way that you have not before. They need time to get used to the new task. Fatigue is normal but pain or numbness and tingling is not normal and an indication that something is wrong.
The reason I ask if you are taking lessons is that a professional pianist will be able to correct bad posture and that could be causing your symptoms. Even thought you are not experiencing pain now you might be stressing your thumb in a way that will cause a more serious problem over time. Also, thumb issues specifically can be a sign of carpel tunnel syndrome which is very common among programmers. I have mild CTS in my right hand and it affects the thumb and first two fingers when it acts up. The pinched nerve makes my hand feel like it's under pressure, it doesn't hurt or tingle. Pinched nerves can often feel like stiffness (sciatica feels like a pulled hamstring). So, in your particular case the adding piano to programming might be pushing your hands over the edge. If you are older it could be a sign of arthritis. Almost all of us will get it at some point.
People on this site cannot and will not give medical advise and even if we could (there are probably doctor-musicians here) it would be impossible to tell based on your question. You should see a doctor about the thumb regardless of your piano playing as your computer work may be causing CTS.
Then, take lessons.