I've been playing around with the pitch changer in Audacity. I noticed that when I change the key on instrumental music, while it may sound off, the timbre is mostly the same. In fact, it only sounds weird for pieces I've heard before since it conflicts with the key I remember; for new pieces, I can transpose it by half an octave and it will sound pretty normal. On the other hand, transposing a song with vocals by even 1 or 2 semitones will make a huge difference. Why is the human voice more sensitive to changes in pitch?
Another thing I've noticed is that in some cases, female voices modulated down a few semitones sound reasonably close to male voices and vice versa, but for other singers no amount of pitch change can make them sound like the other gender. For example, Taylor Swift (song: "You Belong With Me") down 3 semitones sounds somewhat like a tenor. On the other hand, Adele (song: "Someone Like You") down 1 semitone sounds like herself, down 2 semitones sounds in between male and female, and anything 3 or more sounds hideous. Train (song: "Hey, Soul Sister") up 3 semitones could plausibly be female. Elton John (song: "Can You Feel the Love Tonight?") still sounds male up to 3 semitones up, and anything more starts to sound chipmunk-like. What qualities in a singer's voice determine whether they sound like the other gender when transposed or not?