Generally speaking, p.a.speakers have a wider sound spectrum than, say, guitar speakers. They need to be able to produce both high and low pitches equally. So your drumkit maybe isn't giving it enough boost at certain frequencies, and a mixer may be the answer. Except that you probably only have two channels - left and right - that you can plug into that p.a.amp/speaker, and that's made worse by the fact that all that is 'monoed' due to only using one speaker! Although there will be tone controls on that mixer. A graphic equaliser may help, as that can boost certain frequencies more accurately. But where it would go in the chain - you'll have to experiment. I'd say between brain and amp., but bear in mind the g.e. will influence any other sounds going into it.
Another thought - on my Yamaha and Alesis kit, each drum/cymbal has its own volume control, after getting into the menu a bit. Your, presumably Yamaha kit should also. Time to read instructions!