Say I have the waveform of instrument 1 (bass guitar for example) and it contains the frequencies typical of a vastly different instrument, instrument 2 (alto flute for example). In principle, is it possible to process the bass guitar waveform using filtering, attenuation, boosting, algorithms, etc, to produce an approximation of the flute?
If so, is this called synthesis? If not, why not?
By the Fourier theorem, every sound can be decomposed into a sum of pure sine waves. Finite duration or non-repeating sounds require summing an infinite number of sine wave to perfectly reconstruct, but you can get arbitrarily close with a finite number of sine waves.
You can break down the waveform of any noise into its constituent pure frequencies. You can also generate any waveform by combining only pure frequencies. So long as the frequencies needed to construct the second sound are present in the first, then you can do it. If you don't have the necessary frequencies, you can compress or stretch the original waveform appropriately until you do.
Note that this is primarily a theoretical argument - if you have even a single sine wave, you can create any possible wave form by copying, modifying, and combining waves. Whether this is actually practical or in any sense worthwhile is a different story. In principle, though, any sound can be decomposed into sine waves, any sine wave can be turned into any other sine wave, and any sound can be synthesized as the sum of sine waves.
I don't exactly disagree, with Hoagie's answer, but fact is, that the original waveform is not of much help in case of such different instruments, where the waveforms exhibit no similiarity.
You save nothing compared to starting from scratch with adding sine waves in the appropriate ratios, which I consider as strict form of synthesis.
Sort of. The sustain portion of most instrument sounds has a relatively simple, regular waveform which can be manipulated to sound like another instrument. I wouldn't call this 'synthesis', that implies starting from nothing. Modification, transformation perhaps?
But a whole LOT of an instrument's character is contained in the attack portion of the note, often a much more complex, even chaotic waveform. This part of the sound is much more difficult to deconstruct and modify.
Some of the most useful sounds in our toolbox come from the 'hybrid synthesis' era, notably the Roland D-50 and the Yamaha SY range. A sampled attack was followed up by a synthesised sustain and release. The brief sample gave authenticity, the synthesised sustain was very controllable. Best of both worlds.
I’m going to answer a qualified
You’d struggle to make the bass sound like a flute because while it does have a harmonic spectrum that could be filtered in some ways to be similar to a flute, it would be very hard to get the right formant, you’d have to add some noise, and you’d be pretty well stuck on re-creating the transients and ability of the flute to sustain.
It would be even harder to process a flute to sound like a bass - much of the harmonic content of a bass just isn’t present in the flute. And none of the noise and inharmonic partial are available on the flute.
You could probably make a clarinet and maybe an oboe sound like a flute, but not a flute like an oboe.
You can definitely make a bass sound like a piano, and maybe the other way around.
There is at least one process that is commonly called “synthesis” that uses existing sounds to make new ones. That is granular synthesis. That process takes samples of sound and slices then into very short bits called “grains” and then used the grains to make new waveforms. Really you’re not using the characteristics of the source sound at all in granular synthesis, which is why it’s called synthesis.
What does make different instruments sound differently? How do you distinguish, say, a trumpet (sound) from a violin (sound)?
There are two (three) things the ear relies on: volume, but it would be easy to play a recording with more or less volume, so this is a neglectable difference.
The two important differences are:
the intrinsic curve, also called ADSR (attack, decay, sustain, release) is the way the single tone develops over time. A tone starts to sound (attack), stabilizes (decay), sounds out for some time (sustain) and then fades out (release)
the overtone composition is the way each instrument emphasizes (or deemphasizes) certain partial harmonics in a characteristic way. Every real tone a real instrument produces is not only the base frequency but also many partial harmonics of that base tone. Some of these partial harmonics are louder (emphasized) than others and which one these are depends on the sort of instrument which produces the tone. If we talk about a "warm tone" or a "metallic tone", etc., this sum of partial harmonics is meant.
So, - in principle - the answer is: "yes", because - "in principle", as you said in the title, with enough sound engineering (some Fourier analysis and sound synthesis, as @NuclearHoagie said), you can indeed produce whatever sound you want from whatever sine wave you start with. Basically every classical synthesizer does exactly this - to some extent.
If this in principle possible method is - in practice - practical is another question. You can approach the sound of an instrument, but the more musical training one has the better will he be able to differentiate between "almost this instrument" and "exactly the instrument". Notice that the amount of calculation necessary to get it exactly right is exponentially higher than to get it almost right.
This is why modern synthesizers, instead of making sounds from pure sine waves, use samples as base of their sound production. Samples are recorded sounds of real instruments which are then processed.
I think you are focusing too much on the content of the wave package from the source instrument. Look at it this way instead:
If you have these you can just tune the parameters of your model until it sounds nice, for example using some form of gradient descent after turning it into a optimization problem as in machine learning. Step 1/2 can probably be replaced entirely by some fancy embedding, and maybe the ground truth in step 4 can just be some kind of (text, sound) pairs or something plus a model mapping the text to the embedding used in step 1/2 for the target instrument and a model mapping the source instrument to text. Not sure.
I think this is pretty much what would happen if you told a person to play something, then the text would just be sheet music or some language, and the embedding would be whatever your body does to make it move.
Yes. This is what a guitar synthesizer is, at some level. That said, a guitar is very different from a flute. For one example, the note onset is extremely different between a wind instrument and a plucked instrument, so such a synthesizer would need to apply assumptions about information that just isn't there on the guitar to come up with an acceptable flute. The level of satisfaction with the process will have to do with how good those assumptions are, and how they can change from situation to situation.
Getting such synthesizers better might be a good role for AI.
