Classical music, and really, most kinds of music, sound odd when played backwards because the harmonic progression does not lead where you expect it to. Without going to deep into music theory, I'll outline the basics.
A typical cadence, or ending of a chord progression, is a IV chord to V to a I chord. (Roman numerals signify chords built out of a major scale, starting on that degree of a major scale, so V is the fifth scale degree.)
This cadence is a very typical one, and you'll hear it everywhere. What that cadence does, is go from the chord with the most tension in the key… to the tonic, or home chord of the key. In this way, you have tension and relaxation, which is the foundation of harmony.
A more advanced chord progression might go I-vi-ii-IV-V-I. This progression sounds different, but it still ends with that characteristic IV-V-I, also called a Perfect Authentic Cadence.
Now getting into one of the reasons music played backwards sounds so odd: Our ears expect that Perfect Authentic Cadence. (Simplifying here, there are other pleasing cadences.) Sometimes, a composer will do a cadence like: IV-V-vi. This cadence is called a Deceptive Cadence, because your ear will expect the IV and V to inevitably lead to I. When it goes to the vi you get a denial of expectations. While I'm simplifying, these basic ideas of tension and relaxation are what gives music it's sound, feeling, attitude, soul, etc.
When these characteristic cadences and progressions get reversed, it essentially throws that whole system out the window. Now you're going I-V-IV-ii-vi-I and not only does that defy expectations, your brain likely is just going WTF?!? For lack of a better term…
To hear these cadences for yourself play this on the piano:
Courtesy this Wikipedia article:
To hear a recording of just these three chords, go here.
Hopefully this makes sense!