How Can I Replicate The Sound Of An Instrument?

Question

I have experience of basic music theory, guitar and piano. My current project involves creating a computer program that can replicate the sounds of any given instrument (starting with piano).

I can quite easily play the correct notes and chords, but am not sure how to make it sound right.

By this I mean, 440Hz is an A but it does not sound like playing an A on a piano... it sounds fake.

So what other things do I need to look into so that my chords sound like they are being played on a real instrument?

Answer 1

This is an extremely broad topic, so I'm going to try to distill it down to just a handful of points:

There are (basically) two ways of generating sounds electronically: sampling and synthesis. Confusingly, we tend to call all of this sound generation "synthesis", as you are generally working with a "synthesizer".

You will want to read Wikipedia - Synthesizer, but here are some cliff notes:

Sampling is when you literally record the sound of the instrument you want to play into a digital waveform (how all audio is stored digitally), and your software plays back the waveform (or sample), but tweaks it according to how you want it to be played back, either with pitch shifting, looping, or modulation. You can have as few as one sample for an entire digital instrument, or as many as multiple samples for every single note on the keyboard for different dynamic levels or qualities of attack. Nearly all digital pianos use high-quality samples to generate sound, and there are very expensive libraries of orchestral samples that one can purchase if they need to synthesize a full orchestra.

Synthesis: Sampling is technically one type of synthesis. Other types (rather than play back recordings of a particular sound) generate basic waveforms like a sine wave, square wave, and sawtooth wave, often at different frequencies, and add them together while passing them through various filters and other effects to result in a distinct waveform that represents a timbre, or quality of sound. When you are trying to replicate acoustic instruments this way, you call it "imitative synthesis". Usually, though, one would use this kind of synthesis to generate sounds that cannot be generated by an acoustic instrument. Sampling technology is far more effective at replicating acoustic sounds than imitative synthesis.

Based on your question, it sounds like you are trying to achieve imitative synthesis. I'm not sure what you mean by your A sounding "fake" -- if it's at 440Hz, it's just as real of an A as any other A, but chances are it just sounds like a sine wave.

For an extremely simple exercise for the reader, try synthesizing an "organ" sound by playing your 1 amplitude 440Hz sine wave and a .2 amplitude 660Hz sine wave at the same time. It's a long way from there to a piano. You'll want to look at the waveform for a single acoustic piano note (using a waveform editor like Audacity), compare it to what you're generating with your software, and then figure out how to best tweak your software to approach the acoustic piano waveform.

It would also be a good idea to become familiar with some real audio synthesis software first, like Reason, Csound, Max/MSP, or Pure Data. You'll also want to read up on the harmonic series.

Answer 2

You need to reproduce the full spectrum for each note and model the way the amplitude (and harmonic mix) varies over time.

See Analysis and Parametric Synthesis of the Piano Sound

For accurate reproduction you also need to take into account complex effects such as induced vibrations in other undamped strings.

---

Answer 3

This is a complicated topic. Julius Smith has a book about physical audio synthesis. The section on piano hammers has some decently heavy math. In order to make the chords sound like they are being played on a piano, you have some options:

1. Correctly model all aspects of a piano (hammer, string, other strings, soundboard, etc.). This is hard.
2. Record what you want and play it back. Somewhat easier.
3. Model only a few physical effects and get something pretty close.

Answer 4

In case actual sound synthesis is not what you are aiming for, but rather your program is about controlling potentially existing sounds, then perhaps what you want to do is to program using some MIDI API. This way you don't have to create the sounds yourself - instead you can control the sounds available of a software, or of a physical, synthesizer.
If this is the case you'll ikely also want to take advantage of the General MIDI scheme which includes a standardized specification of standard sounds (such as piano, violin, guitar, trumpet...) controllable through MIDI.