5

Say I have the waveform of instrument 1 (bass guitar for example) and it contains the frequencies typical of a vastly different instrument, instrument 2 (alto flute for example). In principle, is it possible to process the bass guitar waveform using filtering, attenuation, boosting, algorithms, etc, to produce an approximation of the flute?

If so, is this called synthesis? If not, why not?

11
  • 1
    The answer is: yes, and the terms synthesis and sound design are what you're looking for. Sep 7, 2023 at 17:12
  • 1
    One obvious limitation: a high-pass filter can make a low pitch into a high pitch, but I’m not aware of a way to do the opposite (at least… not a quick and easy practical one that I can imagine a straightforward approach for). Sep 7, 2023 at 17:34
  • 8
    @AndyBonner Nitpicking but a high pass does not make a low pitch into a high pitch, it only keeps the hight frequencies already present in the original signal.
    – Tom
    Sep 7, 2023 at 17:43
  • 2
    @user45266 - in theory. In practise from a bass guitar, you'd and up with more noise-floor than anything else, if you push it too far.
    – Tetsujin
    Sep 7, 2023 at 17:50
  • 4
    Comparing my answer to others has shown me that this question is unclear. The answer could be "yes" if any processes are allowed, and in that case you could take any noise or even no noise at all and craft an approximation of any instrument and this would be synthesis. If only some processes are allowed, such as only real-time processes or subtractive processes, then the answer is "no" and none of those processes would be called synthesis. Can you please clarify what kinds of processes you're talking about? Sep 8, 2023 at 15:48

7 Answers 7

3

By the Fourier theorem, every sound can be decomposed into a sum of pure sine waves. Finite duration or non-repeating sounds require summing an infinite number of sine wave to perfectly reconstruct, but you can get arbitrarily close with a finite number of sine waves.

You can break down the waveform of any noise into its constituent pure frequencies. You can also generate any waveform by combining only pure frequencies. So long as the frequencies needed to construct the second sound are present in the first, then you can do it. If you don't have the necessary frequencies, you can compress or stretch the original waveform appropriately until you do.

Note that this is primarily a theoretical argument - if you have even a single sine wave, you can create any possible wave form by copying, modifying, and combining waves. Whether this is actually practical or in any sense worthwhile is a different story. In principle, though, any sound can be decomposed into sine waves, any sine wave can be turned into any other sine wave, and any sound can be synthesized as the sum of sine waves.

19
  • 12
    @Steve - the answer is "Yes… but…" Like the answer to "Can I build an ocean-going boat with just an axe & some trees?" is "Yes… but…"
    – Tetsujin
    Sep 7, 2023 at 17:34
  • 3
    A funny consequence of this is that you could theoretically with sufficiently precise signal filtration make every possible sound out of sufficiently-populated white noise... so technically the snare drum can do anything your instrument can :P
    – user45266
    Sep 7, 2023 at 17:46
  • 1
    Indeed. I never claimed it was insoluble… just has rather a large '…but' [I used to work with some of the early physical modelling guys, early 90's… which proved if nothing else if you "Science the sh*t out of it" [Matt Damon, Martian] eventually you'll get there ;)
    – Tetsujin
    Sep 7, 2023 at 17:46
  • 3
    “You can break down the waveform of any noise into its constituent pure frequencies” Ummm… maybe… but how? Are you thinking FFT? Even with that you can’t extract a bass sound from a flute. You’d have to synthesize a lot of content and then it’s no longer really turning a flute into a bass. It’s just pitch tracking the flute and then synthesizing a bass sound at the same pitch. Sep 8, 2023 at 2:05
  • 2
    @Tetsujin: Good analogy — but even beyond that, it seems like saying “Can I build an ocean-going boat from just an axe and a bunch of Ikea bookshelves?” The “but” is all the obvious difficulties, plus that you’d do better by starting at the lumberyard instead of Ikea.
    – PLL
    Sep 8, 2023 at 11:48
10

I don't exactly disagree, with Hoagie's answer, but fact is, that the original waveform is not of much help in case of such different instruments, where the waveforms exhibit no similiarity.

You save nothing compared to starting from scratch with adding sine waves in the appropriate ratios, which I consider as strict form of synthesis.

7

Sort of. The sustain portion of most instrument sounds has a relatively simple, regular waveform which can be manipulated to sound like another instrument. I wouldn't call this 'synthesis', that implies starting from nothing. Modification, transformation perhaps?

But a whole LOT of an instrument's character is contained in the attack portion of the note, often a much more complex, even chaotic waveform. This part of the sound is much more difficult to deconstruct and modify.

Some of the most useful sounds in our toolbox come from the 'hybrid synthesis' era, notably the Roland D-50 and the Yamaha SY range. A sampled attack was followed up by a synthesised sustain and release. The brief sample gave authenticity, the synthesised sustain was very controllable. Best of both worlds.

6

I’m going to answer a qualified

No

You’d struggle to make the bass sound like a flute because while it does have a harmonic spectrum that could be filtered in some ways to be similar to a flute, it would be very hard to get the right formant, you’d have to add some noise, and you’d be pretty well stuck on re-creating the transients and ability of the flute to sustain.

It would be even harder to process a flute to sound like a bass - much of the harmonic content of a bass just isn’t present in the flute. And none of the noise and inharmonic partial are available on the flute.

You could probably make a clarinet and maybe an oboe sound like a flute, but not a flute like an oboe.

You can definitely make a bass sound like a piano, and maybe the other way around.

There is at least one process that is commonly called “synthesis” that uses existing sounds to make new ones. That is granular synthesis. That process takes samples of sound and slices then into very short bits called “grains” and then used the grains to make new waveforms. Really you’re not using the characteristics of the source sound at all in granular synthesis, which is why it’s called synthesis.

4

What does make different instruments sound differently? How do you distinguish, say, a trumpet (sound) from a violin (sound)?

There are two (three) things the ear relies on: volume, but it would be easy to play a recording with more or less volume, so this is a neglectable difference.

The two important differences are:

  • the intrinsic curve, also called ADSR (attack, decay, sustain, release) is the way the single tone develops over time. A tone starts to sound (attack), stabilizes (decay), sounds out for some time (sustain) and then fades out (release)

  • the overtone composition is the way each instrument emphasizes (or deemphasizes) certain partial harmonics in a characteristic way. Every real tone a real instrument produces is not only the base frequency but also many partial harmonics of that base tone. Some of these partial harmonics are louder (emphasized) than others and which one these are depends on the sort of instrument which produces the tone. If we talk about a "warm tone" or a "metallic tone", etc., this sum of partial harmonics is meant.

So, - in principle - the answer is: "yes", because - "in principle", as you said in the title, with enough sound engineering (some Fourier analysis and sound synthesis, as @NuclearHoagie said), you can indeed produce whatever sound you want from whatever sine wave you start with. Basically every classical synthesizer does exactly this - to some extent.

If this in principle possible method is - in practice - practical is another question. You can approach the sound of an instrument, but the more musical training one has the better will he be able to differentiate between "almost this instrument" and "exactly the instrument". Notice that the amount of calculation necessary to get it exactly right is exponentially higher than to get it almost right.

This is why modern synthesizers, instead of making sounds from pure sine waves, use samples as base of their sound production. Samples are recorded sounds of real instruments which are then processed.

3
  • Very well articulated. I leaned a few things from your answer. Thank you (and others) for sharing your knowledge.
    – Steve
    Sep 8, 2023 at 12:45
  • There's more to the attack than can be defined in a volume envelope. See my reply in this thread.
    – Laurence
    Sep 9, 2023 at 11:25
  • @Laurence: I am aware of that and I never said it is only volume. I said the ADSR is how the tone develops over time, I did not say that is only (changes in) volume. In fact, tones usually also contain more or less noise (for instance: slap bass), which also contribute to the chracteristic sound.
    – bakunin
    Sep 11, 2023 at 7:42
0

I think you are focusing too much on the content of the wave package from the source instrument. Look at it this way instead:

  1. can you determine the intended timbre, dynamics, pitch etc from the recording?
  2. do you have a model mapping these states from the source instrument to states of the target instrument?
  3. do you have a model capable of creating or transforming sound based on this target instrument state?
  4. do you have recordings of the target instrument to compare to

If you have these you can just tune the parameters of your model until it sounds nice, for example using some form of gradient descent after turning it into a optimization problem as in machine learning. Step 1/2 can probably be replaced entirely by some fancy embedding, and maybe the ground truth in step 4 can just be some kind of (text, sound) pairs or something plus a model mapping the text to the embedding used in step 1/2 for the target instrument and a model mapping the source instrument to text. Not sure.

I think this is pretty much what would happen if you told a person to play something, then the text would just be sheet music or some language, and the embedding would be whatever your body does to make it move.

0

Yes. This is what a guitar synthesizer is, at some level. That said, a guitar is very different from a flute. For one example, the note onset is extremely different between a wind instrument and a plucked instrument, so such a synthesizer would need to apply assumptions about information that just isn't there on the guitar to come up with an acceptable flute. The level of satisfaction with the process will have to do with how good those assumptions are, and how they can change from situation to situation.

Getting such synthesizers better might be a good role for AI.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.