In addition to the great answers here, I did want to point out one time where key does matter. Not every key sounds the same, so composers will often gravitate towards a particular key to get a particular feel. For nearly every modern instrument, the pitches are chosen according to what is called "equal temperament." This set of pitches let an instrument play in any key, but it turns out that you can't always get it quite perfect. Some keys will naturally have ever so slightly different harmonics than others. These different harmonics add colors to the music that a skilled composer can take advantage of.
The most evident of these appears in any music with a piano. One of the things you want when you tune any instrument is for each note to be twice the frequency of the note one octave below it. However, in practice, real pianos have a slightly inharmonic behavior. The first overtone of a piano is not quite exactly double its fundamental frequency. Accordingly, one tends to stretch the octaves to get them to sound right. This stretching effect causes pianos to have slightly different sounds in different keys, because the ratios between the notes are not quite a perfect logarithmic scale.
Over time, composers learn that certain keys lend themselves to particular feelings that they wish to bring forth. Once they have their opinions, they can often pick a key to compose in based on the feeling they want, rather than having to hunt for the correct key.
As for what a key "is," it's an artifact of what humans have generally found to sound good. Remember, our ears were not designed to listen to music. They were designed to listen to the environment, letting us know where water is or where predators might be. Much of that hardware is still present in our modern day ear, and it leads us to like certain sounds and dislike others. What we have found is that the ear likes some concept of a "root" frequency and the harmonies are built around that note. From there, we find that many cultures have their opinions of what should happen within those harmonies. Classical Western music, for example, has a very clear set of chord progressions that are favored above others because it resonates with what the listener expects (culturally, and with respect to the hardware between our ears).
That being said, these concepts are pliable. Never assume your key or chord progression must be set in stone. Compare classical with modern music and you'll find that modern music contains many more discordant sounds. The modern mind appreciates that discord more than the mind of a 1700's noble did, so naturally the sound has become more populated with them. Modern music still has a concept of a key, but it is more willing to explore what happens when you leave that key behind.