I converted an acoustic, and even replaced nylon strings with steel to add pick-ups, and yet this (fretboard wear) doesn't appear to be an issue at all. There are two reasons for this being so:
First, I filed off my frets instead of removing them, so there is still metal beneath the areas with most stress.
Second, becoming accurate with fretless playing is more of an analog slide process than a digital fret business. You'll become proficient in sliding perpendicular to the strings to achieve your desired note, and adopt that technique or trills (rocking your finger) in place of using bends which are the main culprit in fretboard wear.
Your sound will suffer however if you continue to play with the pads of your fingers. You will have quieter notes with less attack, less sustain, and less succinct tonality. To resemble the tone you had with frets will require playing with your nails pressed against the fretboard, which is no easy trick.
Also your right-hand technique will have to change even if you increase the action. You had some action space between frets before with frets. Now your strings will rest directly on the fretboard, and any plucking or strumming short of perpendicular to the surface is going to result in string slap or buzz.
Wood is softer, and metal would be nicer, but you'll be varnishing the thing anyway. I suppose the stiffer the varnish, the better. I used marine spar varnish and it worked fine. The fine powder from filing my nickel frets got into the wood grain, but I find I prefer the look.