It certainly won't do any harm to mix and match different sounds. Each instrument has different overtones/harmonics which is one reason why they sound different from each other, and sometimes this will fool you into identifying the interval wrongly.
Piano is a favourite, for several reasons - availability, ease of recording, clear sounding notes, etc. Avoid bells and some organ sounds, as these do have many harmonics which will put you off the scent, so to speak. Guitars and violins are good, as both notes can be played simultaneously. Quite difficult with trumpets, saxes and such like, which usually play notes in sequence!