How can you distinguish instruments by their timbre characteristics?

Question

What I am trying to achieve is to distinguish instruments by their timbre. What I have read so far made me think that different instruments have different number of overtones.

Is it possible to identify an instrument from a list of overtones/harmonics for each (or at least some) instrument represented as, for example, frequencies? Or does it depend on the physical state of an instrument?

Or am I going into completely wrong direction and the timbre is not formed, amongst others, with the number of overtones?

Answer 1

A simple list of what overtones are present wouldn't tell you much. What you really want is the relative levels/intensity of each overtone. A list of the overtones with relative intensities for an instrument is called the instrument's spectrum. You might try searching for " spectrum" for the ones you are most interested in. Here's an example for a violin:

Many instruments have virtually the exact same overtones present, and often with very similar relative amplitudes. What is different about them are the relative amplitudes over time. The change in the amplitude of an overtone over time is its time envelope. The combined time envelopes of the different overtones (we'll call it just "envelope" for simplicity) is also a critical aspect of instrument recognition.

Overtones that are relatively stable of a certain period of time are called partials. Overtones that are only audible for short periods of time are usually called transients, and initial transients are important for instrument recognition.

The important aspects of pitched instrument timbre recognition are the envelope and the full overtones spectrum, including partials and transients.

Note that minor components of an instrument's sound (including noises, like breath noise) are also represented in the spectrum, mostly as transients.

Note also that unpitched instruments, like many percussion instruments (e.g., cymbals) or more "noisy" instruments have multiple frequencies present that are not harmonically related and are sometimes called inharmonic components. The frequencies and amplitudes of these less- or non-related overtones creates the spectrum for an unpitched sound and also determines whether a sound is a little bit pitched (like a snare drum). Envelopes and transients are also just as important for unpitched instrument recognition.

Answer 2

For human ears, the relation of the overtone to the fundamental is perhaps not as important as the pitch area that the overtone sounds in. Our ears have evolved to pick out resonance peaks and valleys (see the concept of vocal formants) that are pertinent to distinguishing vowels. An "ah" sound, for example, has an "ah" quality regardless of the fundamental pitch used when speaking or the specific rank of the overtones that lie within the formant areas.

Instruments have characteristic wave profiles based on how the sounds are formed: e.g., a clarinet will at least start out with just odd-numbered harmonics, a string will produce a sound wave with sawtooth wave qualities as the string slips and is grabbed by the bow, a brass instrument will have some pulse-wave qualities as the air periodically forces itself through the lips. But what further distinguishes these instruments from each other is the resonant peaks and valleys that arise from the shape or construction, that are invariant regardless of the pitch being played. The sound is the result of the combination of the "exciter," and the "resonator" for the source vibrations, similar to the source/filter model used for speech recognition and speech synthesis.

And yes, there are also envelope characteristics, especially pertaining to the start of notes that can help us distinguish different sounds.

There is some incredible technology out there now that can pick instruments out of a mix. I am just sharing a bit I know of the acoustical theory. I haven't learned yet how to do the analysis for quickly identifying the harmonic profile. I finally learned the basic math behind DFT analysis, via The Scientist and Engineer's Guide to Digital Signal Processing, but I think it is going to be a long haul before getting the programming and math down for truly identifying instruments.

Answer 3

On a practical/engineering approach, once we have the spectral analysis (i.e. the characterization of the frequency spectrum along time in terms of transients, and harmonic and inharmonic partials, as explained in Todd Wilcox's answer), we need to compare our instrument to a reference database of previously catalogued instruments.

This is done by using a rather large number of spectral properties evaluated from the basic spectrum, for example, just to name a few (there are dozens):

• spectral contrast - a comparison of the spectral envelope with the reference database
• inharmonicity - relative weight of the inharmonic components
• dissonance - comparison of the spectral envelope with a perceptual dissonance reference

The database comparison is done by statistical means, as of course no two samples of the same type of instrument (or even of the same exact instrument) are exactly alike.

Clustering algorithms are typically used: several samples of the same type of instrument are classified into a "cluster" in the database. The sample being analysed is then compared to each cluster, to find where it fits better. Here's an example of the result of such process (the process must be repeated for several combinations of different parameters to obtain good matching results):

This picture was obtained with sms-tools (a set of resources developed by the Pompeu Fabra university in Barcelona), Essentia (a very powerful software tool for audio analysis) and samples collections gathered from Freesound. These are invaluable resources for anyone endeavouring in sound analysis.