If a human is playing the samples, I would prefer it if this latency was consistent, and as close to zero as possible.
Whenever I make my own samples, I zoom right in and trim any 'silence' from the start of the sample. All my samples are saved like this.
I'm not 100% sure if I would notice 100ms of silence at the start of my samples, but I would expect them to play as soon as I'd triggered them. Some smartphone apps of this nature are unplayable because they're not responsive enough.
You can always factor in a quantisation feature if your sounds work best when absolutely synchronised. (delay the playback if triggered early, play sample from just-after-the-start if triggered late, like aligning all the sounds to a 'grid')
When sequencing, it's just as important to have samples aligned correctly too. The easiest way to do this is also to trim any silence from the start of your samples. That is usually all that is necessary.
If you find some of your samples are 'out of time' even with the samples trimmed, then you're either:
Playing the sample at the wrong time. Many sequencers allow you to offset the trigger for sample playback, e.g. not starting immediately, but after a few 'ticks'. This is one way of achieving a 'triplet' when sequencing in 4/4 time.
Playing the sample at the wrong rate. Can be true of longer, rhythmic samples.
Playing a sample that contains leading noise e.g. reverse cymbal, in which case, it's being triggered at the wrong time, you can't add a delay to all the other samples to compensate, you simply have to sequence it to play earlier.
Should you dynamically align the samples? That's very ambitious, you'd need to calculate a known-offset for each sample, either pre-calculated or on-the-fly. It might also take a bit of getting used to for the user, e.g. if I placed my reverse cymbal sound where I wanted it to peak rather than where I wanted it to begin. I guess it's no different than left/center/right justification for word processors.
It might actually be a cool feature, but I wouldn't want it to be dynamically calculated (but an operation to calculate a suggested point might help), I would want to be able to set my own 'alignment markers' to control where the sample starts, since playing a sample like this at different pitches/rates would all require them to be started at different times.