I'd listen to a studio recording of a familiar song, for example, and it would seem off at times but after a while that "off" feeling would go away.
Feeling that a recording is 'off' is not an indication of tone deafness - someone who was tone deaf would typically have no idea whether something was 'off' or not.
Symptoms of receptive amusia, sometimes referred to as "musical deafness", include the inability to recognize familiar melodies, the loss of ability to read musical notation, and the inability to detect wrong or out-of tune notes.
On the other hand, there are some people who have perfect pitch who feel distressed when things aren't what they expect:
Musicians with absolute perception may experience difficulties which do not exist for other musicians. Because absolute listeners are capable of recognizing that a musical composition has been transposed from its original key, or that a pitch is being produced at a nonstandard frequency (either sharp or flat), a musician with absolute pitch may become distressed upon perceiving tones believed to be "wrong" or hearing a piece of music "in the wrong key".
Maybe you're one of those?