This question already has an answer here:
Why do musicians care about which keys they choose?
This question already has an answer here:
Why do musicians care about which keys they choose?
All instruments have technique considerations related to pitch class except for the human voice and perhaps the theremin. These specific considerations then go on to inform other aspects of play.
For example, the piano has white keys and black keys. The white keys are bigger, evenly spaced, and all on the same physical level. Playing in keys other than C major (or diatonic equivalents) requires the use of black keys, which are on a different physical level, are unevenly spaced in groups of three and two, and are smaller. This is a vast simplification, but the ways in which those combinations of keys come together for playing in different musical keys vary in difficulty and comfort for the pianist, depending on his degree of training and experience (but even so, every pianist starts in C major).
Specifically, I know many pianists that prefer the key of B major because it makes use of all of the black keys plus two white keys that fall very easily under the fingers.
In contrast, a woodwind player is typically not going to be very happy in B major, because in general the more accidentals (i.e. flats and sharps) they have to play, the more unintuitive the fingerings are. (Compare this fingering chart for the clarinet C major scale vs. the C flat, or B major scale).
String instruments like guitar, violin, etc. have an easier time of this, to be sure, but considerations are still to be had regarding open strings. Various chords or even styles of music will require the use of open strings which are generally fixed at a specific pitch (or one of a finite set of alternate tunings in the case of guitar).
Just to add to NReilingh's answer, there's also the tonal consideration. The same note played at a different pitch can have a different timbre on many instruments. This can result in a piece of music taking on a different character if it's transposed. For performance purposes, this can sometimes be exploited very effectively to work together with the acoustic properties of the performance space.
On some instruments (such as guitar) the key can also affect what fingerings are realistically possible, which can dictate available chord voicings. Some voicings are more complete or harmonically appropriate than others, depending on the context.
There are a number of considerations:
Perception of absolute pitches
Some people claim to get a different emotional response from particular pitches. So they would experience a different emotion on hearing a middle C, than they would hearing the adjacent D. I believe these people are unusual, but it's plausible that they exist.
Difference in tone between pitches
On some instruments, or in some performance spaces, there are audible differences between tones.
If you play an E on the second fret of a guitar's D string, without muting other strings, the two E strings will resonate, as will the B string. If you play F# two frets higher, you will hear less resonating.
Hence, if you play a melody in E major, without muting, certain significant notes will resonate, which you will not hear as much if you play the same melody in F# major.
Likewise, an instrument's body will resonate at certain pitches. Sometimes pieces are composed for specific rooms, for example a church in which particular pitches resonate.
Other instruments have different causes of a tonal difference between pitches. Your voice takes on different properties depending on where in your range you sing. Reed instruments sound rich in the bass and screechy at the high end. And so on.
Differences in intervals
Most of the time we think of semitones as being equal. From C to C# is the same as from C# to D. However, it's actually not as simple as that. If you make each semitone exactly 1/12 of an octave, then important harmonic intervals don't quite work. A fifth won't match the 2nd harmonic of the root note, for example.
For that reason, a standard piano tuning makes compromises, with some semitones being a greater interval than others. That means that many intervals vary depending on what key you are in. The interval between C and F is slightly different from the interval between D and G.
The difference is very subtle, and can mostly only be perceived as a difference in the frequency of interference "beats" when playing chords. But again, it's plausible that it has a subtle effect on our emotional response to what we hear.
Ease of playing
On many instruments, some keys are easier to play than others.
A novice guitarist can play 3-chord pieces in D using a combination of open chords. A piece in C# major is not so easy!
A novice pianist will like to play in C major, since they don't have to think about black notes. A more experienced pianist will have other preferred keys, since they're no longer put off by black notes, but find that some keys require more convenient hand shapes than others.
Effect of mechanics on rhythm
The choice of key often affects the mechanical things you have to do to reach the notes. An extreme example is the melodeon, in which some notes made by pushing the bellows, and others are made by pulling. In one key, a pair of notes might both be pushes, while in another key, they'll be a push and a pull. The former will be a smooth legato note transition. The latter will involve a short pause and the attack of a new note. So the choice of key has affected the rhythm of the music.
More subtly, for example, on a piano, the change in key would affect the distance a finger needs to move to reach the next note.
Consider a one-octave toy piano, that goes from C to C'. Starting on E, you can play "Jingle Bells" in C major. Starting on A, you can play "Jingle Bells" in F major. But what if you wanted to play "Jingle Bells" in G major? You would run out of notes.
The same issue exists for "proper" instruments, with more complex melodies. Perhaps you can get around it by going up, or down, an octave, but this affects tone.
And of course, this is an issue for the voice. Anyone who sings along to their own guitar, has been forced to transpose, because the melody has gone higher, or lower, than their voice can reach.
In solo vocal music, the key of a song is usually established by the composer/songwriter, but when it is performed or recorded, key of a song often gets changed according to the needs of the singer who is performing it at the time.
As a singer, I have to point out that everybody's voice is a little different, and each different singer instinctively knows if a given song will be easier to sing and sound better if it is played in a higher or lower key. So if you are an instrumentalist, and you bring a certain song to one particular singer, that singer will likely request that you play the song in a different key that works best with that singer's voice.
The key that a singer likes, for a particular given song or composition, is not determined by whether the singer is a bass or alto (low voice) or a tenor or soprano (high voice) per se. Rather, it is determined by the tessitura of the melody of the song. "Tessitura" is the technical term for the range of notes in a song: find the lowest pitch in the melody, and the highest pitch in the melody, and figure out if the song leans more heavily on the higher notes or the lower notes. A singer will request that the piece be transposed to whatever key makes it easier for the melody notes to sit within the range of the pitches that that particular singer sounds the best singing in.
In classical music, or orchestral music, it's usually not practical for the entire orchestra to transpose the piece to suit the singer. In that case, the orchestra director will hire a singer that they know can sing the song well in the key in which it was originally written.
For what we call "art song", classical music usually written for one singer and one pianist, it's common to see the same song published in sheet music in two different keys, one for "low voice" and one for "high voice". Thus a singer will have a choice of a whole arrangement written out in one of two keys, and can select one and give it to the piano player to learn.
However, in pop music, R&B, and particularly jazz, it is very common to hear three or four different singers perform the same song, but each singer will select an entirely different key to sing it in, depending on that individual's voice.
I'm a high tenor, and there is a jazz band that I sometimes "sit in" and sing with, without rehearsal. Most of their song charts are written for a singer with a low voice, so when I get on the bandstand, I turn around to the band and I name the song I want to sing, but I tell them that I want them to play it in a higher key than what is in their sheet music, and I name the specific key that I want for that one song. Jazz musicians can usually handle such a request, "transposing on sight", but sometimes I write out my own chord charts for the song, transposed to the key that I want, and I pass my charts around to the musicians.
Besides the practical considerations (to do with the instruments selected and physical keying), there is one very subtle one to do with tuning.
Specifically, depending on the type of tuning you're using, the exact frequency ratio of a 5th rooted at A may be ever so slightly different than the a 5th rooted at E.
So, the two different frequency ratios (however subtle) still actually produce a slightly different chord. Even being off by a couple of Hz can muddy a chord, so it is definitely conceivable that certain pieces played in a certain key just "sound better".
There have been many great answers so far, and all different which will give you some idea of the scope of this question. It can be to do with
I'd like add that there's a different mindset between composers and musicians in terms of picking the key, because in my mind most composers choose their musical vision over practicality, and most musicians will choose practicality over musical vision.
I would Highly recommend listening to '24 Preludes and Fugues' also known as 'The well tempered clavier' by Bach, all the way through, as it takes every key From C all the way up to B in both major and minor. What's important about this collection is that it's commonly cited as one of the key works in establishing the idea of 12 keys. Listening to the whole thing will give you a sense of all 12 keys, major and minor.
The astounding thing for me about keys is that every one has it's own flavour, it's own subtlety that I don't think you can pickup unless you compare them directly. I hate to be subjective, but I found after spending time learning which key related to which piece in the well tempered clavier, that I could pick up the key of other pieces outside it. A perfect example is of the following 2 pieces in F minor - Opeth - Heritage, Bach - Prelue and fugue no 12 in F minor.
In cliche'd metaphor terms it's like the difference in wines. There is a subtle difference to most people between any minor key, but to the keen eared musician a c# minor song and an E minor song are completely different beasts.
The problem with this way to think of keys is that often it's not practical. why should you up the difficulty on piano from an all white key C major scale to some weird key with lots of awkward fingerings? Some dedicated musicians will, for the sake of the piece, but really it's down to whether you think the technical difficulty is worth the pay-off of a more appropriate key note.
I'm a guitarist, the E minor and A minor pentatonic's are my friends who I can always count on. When I'm playing guitar they are my home ground and stable point of reference. When I compose however, my focus shifts to what key sounds best, what note sounds best, regardless of how hideously difficult that particular note might be to actually play!
When I compose I make things difficult on the musicians, but when I play I make things easy on myself.