I have thought that the reason to sing for ear training or with various drills is because it taps into a latent human ability to match pitch, to hear a pitch and then copy with your voice box. If a person isn't made to feel self conscious - like in an ear training test - or hasn't been defeated by a society that discourages singing for all, it isn't a big deal for most people to simply match a pitch.
But if you need to match the pitch on an instrument you first need to know some technical stuff about how to work it and then you need to control it too. Imagine, for example, if someone sang middle C, then handed an untrained person a French horn and asked them to match the pitch! They aren't going to be able to do it, because the instrument is a barrier.
If I match a pitch vocally, I don't need to know what the pitch is. But, by necessity, on an instrument I have to be aware of the pitch, or at least the fingering, technical execution of it. Even if you aren't a skilled singer and your a bit off pitch, you correct it quickly. There seems to be a neurological "wiring" in the human brain to get the voice to match what the ear hears. You really just do it. You don't consciously manipulate your vocal chords. You don't get that connection with an instrument. If you don't have perfect pitch, or aren't highly trained on the instrument, you have a lot more groping around to find the pitch.
It seems to me a big difference would be the need to give a reference pitch.
For example, someone could ask me to do a number of things vocally, in call/response fashion copy the teacher singing a melodic interval, sing a certain interval above the teacher's tone, sing the third of a chord the teacher plays, etc. All those things could be done without any reference to specific pitches.
On an instrument similar things can be done, but first start by giving an _specific reference tone. You might say "we are starting on D" and then the teacher sings the melodic interval. The student knows the first tone is D, but the point is to hear the relative relationship of the second tone and finger the appropriate interval. Some basic technical training for intervals on the instrument would be needed, but I don't see how that can be avoided. The idea is naming the pitch of the starting tone, the focus can shift to the relative relationships of tones. If you don't have that starting pitch, it's like expecting the student to have perfect pitch and relative pitch.