You have stated what you mean by 'musical' - that it reflects a 'positive opinion' of listeners with background in Western music. That criterion could be used in listener studies by asking participants about their preferences to various musical examples. I am not aware of any such study to date, although it would certainly be a testable hypothesis (and not, at the moment, an established fact).
Vibrato is obviously not what you refer to. But take a look at any synthesizer, and you'll find a portamento control. Also, in the performance practice of romantic music, musicians were sliding a lot more between pitches than later became the norm. Usually, portamento playing still would land on pitches for sufficient long time to make them stand out. Perhaps what you have in mind is a situation where no distinct pitches can be picked out by a listener. If your claim is correct, that stable pitches are prefered to constant glissandi, then an explanation might involve the cognitive aspects of pitch processing and memory for pitches, but that's speculation.
For some more conceptual framework, you may find Pierre Schaeffer's Traité des objets sonores useful, and maybe even more so Wishart's On sonic art. Wishart talks about the lattice, the fixed mesh that pitch and rhythm is locked onto in some music, as opposed to dynamic morphology which is akin to gestures, where musical properties are in a state of change. Often there is a lattice (pitch scale) that is ornamented with dynamic changes (vibrato, portamento).
It may be true that most musical traditions so far have been lattice-based, but if you include a few non-western traditions and late modernism you will find plenty of examples where the lattice is more or less absent. Look up James Tenney's For Ann (rising) for starters. Do listeners dislike that? At least some composers have been curious enough to explore the idea.