1

Sorry if this question is a bit low level into the programming, but I am wondering how to build a synthesizer of realistic sounds (without using samples). In the beginning, just basic instruments (guitar, drums, bass with effects), then to voice and nature sounds like lightning, fire, crickets, flowing water, etc.

The way I see it, I would start with the low level oscillators (which I know very little about yet, but this question is the start to a journey of learning audio by doing, I just need some higher level motivation), and add effects to them. But then the second step is to define all types of sounds instruments can make.

I don't really know what that means, but this is where I'm at... There is a pound sound like the drum, there is a sustained but lessening sound like the sound of a plucked string instrument. There is the sustained sound of a note in a flute. Then second to this "category" of waves (it seems to call them waves in this category), there is the transition between notes in an instrument, like sliding on a violin, vs. striking a single note.

So I guess it boils down to waves and wave transitions then. Is there a list of all types of sound waves somewhere? Something like that? I know I've seen the 4-ish standard waves of an oscillator (sine wave, triangle, etc.). But what about realistic sounds? How many types of waves are there? Can they be categorized?

If I knew how many types of waves there were and categories, then I would have a foundation upon which I could build all other sounds. I could start by defining lets say "The 100 Basic Sound Waves", and then I could play them at different intensities with different notes with different effects and durations.

Has anyone done this sort of work already in collecting the wave types to make arbitrary sound from a reasonably large alphabet of symbols?

Note: I was wondering if I should ask on Sound Design Stack Exchange, but I realize I like the idea of associating this more closely with music theory and real instruments (because I used to play in a band). Asking here you might know the higher-level building blocks on top of your standard low-level audio API, so could point me in a better/faster direction.

In terms of "all instruments", I mean every abstract "instrument", from a guitar, to a voice, to a hammer, to a keyboard typing to a flame to water to any thing that can be used to make sound. What are the categories of sound types that can be made?

Sort of like asking "what are all the shapes of 3D objects", and it boils down to shapes with certain number of sides with concave and convex. Something simple and elegant. But you can get more complicated and deep with it if you want, it evolves out of some basics.

Improve this question
  • 4
    Apart from the fact that this is way too broad, you are starting this from the wrong end. First you need to define your models - string, plate, tube; density & stiffness; how it is excited, continuous or impulsive, then what resonances are applied post-excitement. Once you've got that little lot figured, then you can start to think about articulations. – Tetsujin Jul 22 '20 at 7:28
  • 2
    Look up 'physical modelling' because that starts ground up, rather than top-down. – Tetsujin Jul 22 '20 at 7:30
  • Where are the categories though, I would love to find lists of the records in each category. – Lance Pollard Jul 22 '20 at 7:33
  • I would like to find something other than straight physical modeling. Some sort of short cut that gets you close to real sounds but not quite. Sort of like a 2D cartoon game vs. 3D realistic life game. But not 8-bit sounds that's too limiting. – Lance Pollard Jul 22 '20 at 7:37
  • 1
    The categories start from the bottom - string, plate, tube etc. By the time you eventually reach the top, you are into articulations, which is a set of variables on top of the pyramid. You don't start from the top of the pyramid. Such as a bend, slide or gliss is completely different depending on which model you started with. – Tetsujin Jul 22 '20 at 7:37
6

Interesting question, and probably one that has concerned anyone who has tried to write a physical modelling or additive synthesizer - one that is 'in theory capable of every sound' - because of course, 'everything' is hard to work with. How do you categorise, parameterise, and control the sounds if your starting point is "everything"?

I'm going to stick my neck out and say that I don't think there is yet a widely-adopted method of categorizing all types of sounds to the level of detail that allows you to think about recreating them realistically (I'd love to be proved wrong!). I can think of things that have dealt with a small subdomain of sounds - for example, https://en.wikipedia.org/wiki/International_Phonetic_Alphabet that deals with spoken sounds.

Some things you could take inspiration from -

  • Organology, and particular classifications such as Hornbostel–Sachs

  • As suggested in the comments, you could look at particular physical models used by physical modelling synths - 'plucked string', 'blown tube', and so on. Of course there's no one way of categorizing these either - some PM approaches might see 'string' and 'plate' as different models, but you could see them as variations on the same theme too.

  • SAOL, which attempted a somewhat general model of how sound could be made

  • You could also attempt to classify spectra, rather than focus on the way in which sound is produced - e.g. you could consider sounds with even harmonics only, sounds with odd harmonics only, sounds with all harmonics (in different proportions)... and then go on to consider sounds that don't follow the harmonic series, such as noise spectra and so on.

  • Googling 'Audio classification' currently brings up various AI-based approaches. If you have any interest in AI, you could see if you could find an AI-based approach that created classifications that were useful for synthesis.

In a universe where people knew a lot more about sound and synthesis than they do now, you could imagine a scenario where there was a general, abstract way of describing sounds, and then that could be input into any type of synthesizer, each of which could attempt to reproduce that sound as best it could. Your question certainly makes sense conceptually, but you may be a bit too far ahead of the curve....

Improve this answer
  • FWIW, I have no interest in statistical AI, but everything you said is great, leading me in all kinds of directions! – Lance Pollard Jul 22 '20 at 11:52
  • 1
    @LancePollard I think your 'building block' approach is an interesting way to think about synthesis. One question - you said you don't want to use samples, but why not? I mean, you might not want recordings of identifiable, particular instruments - but coming up with a set of sample data that represented different building blocks would seem to be in the spirit of your approach. Sample based synths did sometimes have voices programmed with the attack from one sound, and the sustain from another - but you could probably do better if you carefully created samples that could dovetail together. – topo Reinstate Monica Jul 22 '20 at 14:47
  • Those are good points, maybe I will consider samples at some point. As I'm in the beginnings of learning about this, I would like to see how far you can go (and what it takes) to do everything purely programmatically without any data. – Lance Pollard Jul 22 '20 at 15:17
1

You are asking about the Fourier Transform. It translates functions into a series of sine waves. I recommend you learn a bit about calculus before delving in though. If you are programing you'll be interested in MATLAB's DFT and FFT: https://www.mathworks.com/help/signal/ug/discrete-fourier-transform.html

Waves such as square waves can be constructed using sine waves: https://www.khanacademy.org/science/electrical-engineering/ee-signals/ee-fourier-series/v/ee-fourier-coefficients-for-square-wave https://www.khanacademy.org/science/electrical-engineering/ee-signals/ee-fourier-series/v/ee-visualize-fourier-series-square-wave

Visualization of this:

You may get varied results depending on which instruments you use because some instruments produce sounds which are almost entirely the overtone series. For example, you would probably need more waves to "render" a bassoon, which is almost all overtones, than a trumpet.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.