When recording, the backing track is almost always put down first. One reason is that it provides the tempo for the piece. If you can't keep time with a backing track, imagine what a disater it would be if you recorded just your part first !!
So, whether it's vocals, solo trumpet or flute, that usually is the last to be put down.
Given that the backing track has been recorded with a click track, which it often is - to keep everybody together in time - then your problem is that you're not internalising that tempo.
Using a metronome supposedly works - lots of players swear by them. Personally I swear at mine, as they're soul-less, and unforgiving. Much prefer a simple drum track , as it's more like playing with at least one other. Give it a try, bearing in mind that you, like many others, slow a little on the lesser known, more difficult sections, and do the opposite on the easier, better known ones.
To improve, try starting the metronome, getting the rhythm inside you, clapping, clicking, whatever, going out of earshot, and returning to check how your timing is. Same idea with any pop type song on the radio, lock into the beat, turn down the sound, keep it going, and check a few seconds later. As you get better, make those few seonds into longer periods.
For more practice, tap out the notes as you would play them, one tap at the beginning of each, while the backing is playing. Use your body - shrug shoulders, tap foot, nod head, to keep in time. Most people will stray at some points, so it's not incurable! Also, there are many tracks that don't keep exactly to the given tempo. You can check that with your metronome - set it exactly right at the beginning of a song, and see how the song tempo varies. Partly that's the music being portarayed, and sort of taking over, sometimes the drummer will push a piece to make it sound more exciting.