Lots of classical music words usually have a specific language as their origins (mostly Italian), so, unless an extended literature can confirm standard spelling practice that altered the original word in other languages (that's how languages evolve), the original spelling is to be considered as the proper one: in this case, Italian.
Even the sibling Italian Wikipedia page (or any standard translation) of Cello would report the correct spelling: Violoncello.
Music is full of "loanwords" (as standard terms borrowed from other languages), most of them are Italian, some French or German, and almost all of them share a Latin root. What you should look for is for:
- actual (and most common) origin of the word;
- current spelling in that language:
- possible standardization and adaptations in other languages;
- "Ouverture" (from French), usually leaving out the "u": "Overture";
- "Adagetto" (Italian slightly diminutive form of "less slow than Adagio"), commonly written as "Adagietto"; note that the modern italian rule says that -gi (or -ci) becomes -gie (-cie) and not -ge (-ce) only when the g is following a consonant and when the i is not accented, but there are still some words that ignore that rule: valigia (suitcase) -> valigie, camicia (shirt) -> camicie;
- "Leggero", often written as with the antique form of "Leggiero";
- all extreme augmentatives, like "Fortississimo" or "Pianississimo": the italian rule says that the augmentative (-issim*) should always be added in full form, so the correct forms should be "Fortissimissimo" or "Pianissimissimo", but the reduced forms are not unheard even from italian speakers (I heard a lot of them in the Italian speaking regions of Switzerland, but they also have their own version of italian for many words);
- lots of plural forms (like "Tempo") that use the original word extended by the language rules ("Tempos" instead of "Tempi");
While some of the above terms and variations are now considered almost standard convention, a proper score should try to follow the original language those terms come from, unless extensive usage is now considered as proper for the intended writing language of that score: if you want to write expression markings in German language, you should follow the musical german convention (consider Wagner markings, for instance); this means that if you're writing text markings in English, you should use Cello.
So, if your writing uses a German notation text, you could use "Violincello". Otherwise, "Violoncello" (or "Cello", which is not uncommon even in italian) should be the preferred form: this considers what correctly pointed out before: the "original" instrument is Viola (etymology: 'Vi-tu-la -> 'Viu-la, with the accent on the first syllable), with the Violon[e] as the bigger version of it, and the -cello suffix used to indicate a slightly smaller version of the bigger one: Viola -> Viol-one -> Viol-on-cello. The -*ello suffix doesn't always have a "c", and it depends on usage, but it mostly indicates a lighter - and often "funnier" - version of the original word (similarly to -etto, like "allegretto").
This is pretty common in Italian: take for instance the "Calzone" ("big sock" or "big trouser") that is a folded version of pizza, and often becomes a "Calzoncello" to indicate a slightly smaller version of it, but not as small as the one commonly known as "Calzoncino": a big sock/trouser ("calza"/"calzone") that's considered standard, made smaller but not enough to be a "normal" sock/trouser.
There are other words that follow similar rules, such as "venticello" (small wind), "orticello" (small home kitchen garden), "alberello" (a small tree).
So, yes, it's just "Violoncello". As a "Violincello" would indicate a slightly smaller version of a violin, but probably bigger than a "small violoncello". Which could also indicate a "Violoncello piccolo".