sound can be represented with an oscillating wave...
It's true for a pure tone (e.g. an electronic beep), but musical sounds are always made of multiple waves, theoretically an infinite number of sine waves.
Musical instruments actually produce complex sounds which are a distribution of pure tones with different frequencies and amplitudes. This distribution model, referred to as timbre, is particular to each instrument, as it's actually the result of a distortion created by the instrument. When playing a A, the 440Hz pure tone is actually played along with various additional waves, all multiple of 440Hz, called harmonics. The exact distribution varies from one instrument to the other, hence the sound heard is different. I'll explain in detail.
Notes would be pure tones only if instruments were able to create perfect sine waves, it's not case. It's very fortunate, as this is a pure sine wave (but your computer will still create a bit of distortion).
It could be produced when two pieces of metal clang together...
I'll comment this only briefly here as it is a different topic. For percussion instruments, except for a few ones, of which timpani and xylophone, the sound has no definite pitch. The sound is not even a periodic wave, it doesn't repeat at regular interval. Such sound which is a random mix is known as acoustic noise.
Pure tone vs complex tone
A sound can be anything, and in some cases it can be a pure tone, a sound made of a single perfect sine wave:
Pure tone: Time domain view
A diagram like above, where the horizontal axis is time, and the vertical axis shows how amplitude varies with time, is appropriate to see the shape of the signal. There is another useful view, where the main axis is the range of observed frequencies. This is called a spectral view (a view of the audio spectrum). On such a diagram, a pure tone appears as an isolated peak at its own frequency, and the rest of the spectrum is flat:
Pure tone: Frequency domain view
As long as the sound is steady, these two views are equivalent, they show exactly the same information.
One might think: The spectrum view shows only the frequency and amplitude of the signal, not its shape, maybe it's a square wave, who knows? Actually if it were a square wave there would be many additional peaks in the spectrum. The spectrum really tells us all the smallest details of the shape as we'll see further down.
Hence, the short answer to your question is three-fold:
Sounds from vibrating devices tend to be sine waves, two instruments should therefore create the same sine wave sounds. It's not true, it means the shapes of the waves have been altered, and there is a cause to it.
The difference is visible on the spectrum, by looking at peaks. I'll show further down informative spectra of the same note for different instruments. Alternatively the difference can be seen by looking at the shape of the waves, but for complex sounds this is not practical, nor very informative. Nevertheless, as soon as the spectrum / the shape is different, the sound is different.
This thing that changes waves is known under the general wording distortion from an engineering point of view, and timbre from a musical point of view for which distortion is actually desired. Each particular instrument distorts sounds according to its timbre, adding peaks into the spectrum.
So let's dig into these two aspects: Wave spectrum (next section) and timbre (one more section further).
Harmonics: Mathematical proof
The most common distortion, which is at work in musical instruments, is related to the amplification of the perfect sine. The wave can be clipped because its amplitude grows beyond the capabilities of the amplifier, resulting in the top and bottom of the wave clipped at a certain level:
Clipped sine wave: Time domain view
This sound is still periodic, and the frequency is still the same, but it's no more a sine wave.
It can be proved mathematically any complex sound which repeats its own shape at frequency f, regardless of how chaotic this shape is, is just the sum of perfect sine waves of frequency f, 2f, 3f, 4f, etc, each with its unique amplitude.
Indeed our clipped sine wave takes up more space in the audio spectrum than a perfect sine wave:
F is the fundamental frequency (first peak above), the one the signal repeats itself over time, and 2f, 3F, 4f, etc, are the harmonic frequencies.
The shape equation and the set of sine waves are mathematically the same thing, the two sides of the same coin, we can look at a sound either way, none is more correct and they are physically indistinguishable. As soon as a sine wave is distorted, its spectral view is changed and the harmonics distribution adjusted to match the new shape.
This conversion from the shape view of the wave to its equivalent spectral view is mathematically done using a Fourier transform (FT), the computational method to do it is the fast Fourier transform FFT.
There are lots of web pages where sine harmonics materialized by fixed-length arms rotating at constant frequencies just combine to reproduce any hand-drawn shape.
Now we know harmonics are the result of sine distortion, let's see the origin of the distortion.
Harmonics physical origin: Instrument timbre
String vibration and other oscillatory phenomena are naturally sinusoidal. It cannot be different. The reason is when a string or another vibrating element is moved, it develops a restoring force which is in theory proportional to the displacement, the resulting motion is a sinusoidal oscillation (principle of the harmonic oscillator). It's like the trajectory of a planet orbiting its sun: It has to be elliptical else either the planet falls on its sun or it escapes its sun.
However if this sun is not a perfect sphere, with a perfectly symmetrical density, then the gravity field is not equally distributed anymore and the perfect elliptical motion is distorted. This is the same with a sine wave, its shape is influenced by the material while transmitted, and most of time, the perfect sine is distorted.
By itself, a piano string may generate a not so distorted signal close to a sine wave. Note this sound is weak, and is not the one heard by the audience. The string vibration is transmitted to the wooden soundboard which role is to amplify the sound and make it audible.
Wood is not a perfect material for sound transmission, technically we will say it's not a linear system. Because of this non-linearity, wood will transmit the wave differently, e.g. instead of transmitting the dashed blue sine which is a perfect amplification of the input, the amplifier creates this orange wave where the gain depends on the amplitude:
The non-linearity of the amplifying element distorts the sine wave
The way the soundboard affects the string sound, or the saxophone brass body affects the reed sound is unique, due to the unique physical characteristics of the instrument. This is called the instrument timbre. Timbre in music domain is the same than the impulse response in audio engineering. A piano timbre is comparable to how a room creates an echo, a temporal distortion. Instruments with different timbres (different impulse responses) transmit differently the original sound, exactly like different rooms show different echoes, transmitting the sound differently.
Different piano makes have different timbres due to the choice of materials, the design of the parts, and many other details. Timbres can even have differences among the same design, because all trees are not equal.
While a computer beep has a simple timbre (close to a single frequency spectrum), musical instruments have elaborated timbres with various harmonic amplitudes. See these spectral views for the same note (A of octave 4) from a tuning fork, from pianos and from a saxophone:
A4: Frequency domain view. Stolen from this question
Note also the initial wave created by a guitar string is already different from the original wave created by a piano string, because even simple strings are non-linear systems and therefore distortion occurs and harmonics appear.
For a string to be a linear system, it should exert a restoring force strictly proportional to its displacement when moved by the performer. Unfortunately for the science, and fortunately for music, such perfect elastic material doesn't exist, and the restoring force becomes larger than it should as the amplitude increases.
Harmonics perception by our ear
Our ear is not very reactive to a sound which is not a sine wave. We do not sense the sound as a whole because of its chaotic shape. It's strange when thinking at it, but our ear actually detects individual harmonic frequencies present in the sound. Our brain immediately makes a distinction between a C played on a piano and the same C played on a violin, due to the different harmonics distributions (the different timbres).
However while our brain cares to process harmonics, it keeps the result for itself, it's actually difficult to voluntarily hear individual harmonics (see comment by Ian Goldby). It seems strange we cannot easily isolate the different harmonic frequencies from the whole sound, because on the other hand, we are perfectly able to distinguish two notes which are not consonant, e.g. D and A. Harmonics consonance hide them to us. Still our brain is nice enough to share with us identified dissonances, frequencies occurring simultaneously but which are not multiple of some other frequency.
One more strange thing about what we hear: If the fundamental frequency is artificially removed from the spectrum, our brain, which has been trained to perceive the relationship between a fundamental and its harmonics, will just recreate the missing fundamental and process the sound as if it were complete.
Chords are based on their harmonic spectrum
Harmonics are fundamental (if I may say) in music. When notes match, we just say they are in harmony, or they are consonant. This happens when harmonics overlap.
Notes C E G are played simultaneously are consonant. Their frequency is in the ratio 4:5:6 (they are all multiple of some frequency f). The triad sound contains:
- C note 4f and its harmonics 8f, 12f, ...,
- E note 5f and its harmonics 10f, 15f, ...,
- G note 6f and its harmonics 12f, 18f, .....
CEG triad: Frequency domain view
We see most of the harmonics overlap in some way. This is why they are perceived as a whole and are harmonious.