I'm not a huge classical music fan, but I think your suspicion is probably unfounded: Average resting heart rate is in the mid 70s for most people, which is not outside the range of classical music (Adagio - Adagietto).
Heart rate can go up just as a psychological response (excitement, nervousness), emotions which many pieces of music are designed to inspire. Plenty of rousing military-themed pieces have sections where the temp speeds up as the excitement builds, which may be (intentionally or unintentionally) related to heart beat.
Heart rate can also go up quickly with any moderate activity, for example dancing. A reasonably fit person might have a heartbeat between 90 and 110 bpm when dancing a waltz, which is close to the tempo of the music (waltzes are often around 100bpm).
Another physiological reason for a specific beat rate might be that music is related to the speed at which people dance - some speeds are more comfortable than others, depending on your mass/inertia, agility and strength. If the music is too slow, you might have balance problems (need to put your feet down too soon). Too fast, and you just can't keep up (although some people seem to manage). A fast Lindy hop (ok, this isn't classical, but the principles will still apply) might be 150bpm. That's really fast, for even a really fit, agile human. If you go to a Drum and Bass gig (~160bpm), you'll notice that everyone is actually dancing pretty slowly - at half time, 80bpm - because it's basically impossible to dance at 160 for more than a few seconds in any enjoyable way (unless you're jacked up on something, perhaps).
It's also worth noting that your breathing rate is affected by your pace - when walking you usually breath once per step, when jogging or running, this might go up to 2 or 4 times per step, depending on how fit you are. The same happens while dancing, and that will affect what tempos are comfortable for dancing.
Not that not all of this is necessarily conscious design on the part of the composer - there's an evolutionary process at work in music. If a song is enjoyable (for what ever reason, many of which will be physiological), it will be more popular, and more composers will be inclined to emulate it. That's a major part of the reason that most pop music doesn't go much outside the range of 80-110bpm.