This question is inspired by comments to this related question:
I would be surprised unless you could find a mobile app or at least some Apple/PC SW that filters out a given instrument from the orchestra sound.
Is this possible at all?
Music: Practice & Theory Stack Exchange is a question and answer site for musicians, students, and enthusiasts. It only takes a minute to sign up.Sign up to join this community
No it is not possible given the current state of art. Therefore recording engineers distribute microphones to have separate tracks for the instruments or instrument groups (so they can change the balance) and therefore special recordings as music minus one are required with filtered soloist voice to produce training material with full tutti voices and nothing else.
Artifical Intelligence may provide that possibility in the future, but I would not hold breath.
Note that splitting to all components is even more difficult, than to extract one instrument.
No, at the present state-of-the-art, 'un-baking the cake' isn't practically achievable except in special cases like a stereo recording with ONLY one instrument being panned centre etc.
However, I would be very surprised if the military aren't interested in software that can isolate one voice from a babble. If that is successful, stand by for a spin-off.
Try this! If it can recognize a vocal...
There are machine learning models that attempt to separate recordings into several instrument tracks- notably, Deezer's "Spleeter", MIT's PixelPlayer, and Izotope's (very expensive) RX8. Researchers "teach" their machine learning program by feeding it large amounts of real multitracks. After training the model, it can be given any recording, and it will attempt to create multitracks that "look" like the ones they learned from.
It's far from perfect, and each instrument track will have some extra noise and significant pieces missing. The software can't always tell the difference between, for example, a hi-hat or a singer making a "t" sound. The result ends up sounding like a very low quality mp3. Still, it's an interesting proof of concept and will almost surely improve over time.
Right now, Spleeter can be installed for free, but it only comes with pretrained models for pop/rock-like music (Drums, Bass, Vocals, Piano, Other). If you want to separate an orchestra, you'd have to find yourself lots of orchestral multitracks to train it on.
MIT's demonstration of their model:
Demonstration of the Spleeter model:
That depends on how clean you expect it to be and how automated you expect it to be. Technology exists to identify specific voices in a mono recording of several people in a noisy place so I can't believe that it is impossible. But you would have a lot of heavy lifting to do and some manual labor to decide if what the filters are catching is real or an artifact.
There exists s/w, and homegrown versions are not hard to make, that can ID the separate notes present in a recording. But the next step would be to attribute them to a specific instrument. To do this one would need to ID that a specific instrument exists from among the collection of sounds. This is not impossible as we can take recorded versions of impulse responses for each instrument, or use simulated data from models and convolve this with the recording. I disagree with the comments that it's impossible, or not even close. But there are numerous pitfalls and possible ambiguity. It would be an easier task if rather than trying to prove that there is a violin in the mix of 100 instruments you were to provide a library of instruments that are known to be in the recording. Then you've reduced the effort required and the number of errors that could be made.
Even so, you will have overlapping sound from many instruments mixed together and there will be possible ambiguities and misfires w/r to identifying which note was played by which instrument. Even if you could do a crude separation that got the notes correctly ID'd and mapped to the instruments the actual sounds could be a jumbled mess as you would likely have gotten the amplitude and phase distribution for each instrument wrong. Say a violin and Sax are playing the same note and you correctly discovered both instruments based on detailed harmonic profiles. How do you determine if they are both contributing 50% to the overall sound? Also, if you found the correct peaks in a signal filter but didn't get the phase and subtracted out what you thought was the Sax the "violin" might be distorted as a result of some phase ambiguity. You would need to correct this by then synthesizing the original track using the separated signals, and modulate the unknown variables in each spectrum until you succeeded in match the original noise free. And even then you may have a whole space of possible mixtures that correctly synthesize to the original.
Again, I assert, not impossible but very costly to get a dirty result. In the end it may not be worth anything.