Is there a way to strengthen your mouth muscles specially to blow brass instruments without mouthpiece?
You really need to get more specific than "mouth muscles". The formation of your mouth and lips that is used to create a tone on the mouthpiece is called your embouchure (colloquially referred to as "chops" by many musicians). Once you identify what this is and what it feels like, as well as which muscles exactly are being used, it basically amounts to forming an embouchure (without the mouthpiece) and then training strength and endurance of that embouchure like it was any other muscle group.
What may be difficult, however, is if you are unable to form an embouchure without the aid of a mouthpiece. You need to be able to do this first.
Freebuzzing is the practice of buzzing tones with your lips without the aid of a mouthpiece. You may have already practiced buzzing tones on the mouthpiece only without the trombone; this is just one step further. A common exercise is to play a note on your instrument, sustain that note while separating the trombone from the mouthpiece, then sustaining the same note while removing the mouthpiece from your face, then finally going in reverse until you are playing the note on your instrument again. I would do this on a top-of-staff Bb or middle F, holding the trombone and slide with your left hand, and holding the mouthpiece steady with your right. (You will need to loosen it enough that the trombone can be removed from it without any friction, but keep it steady enough that you don't lose any air before you remove the instrument.)
The above exercise can be done in many variations, but the purpose is always to be able to sustain your embouchure (and tone) regardless of the amount of physical or air pressure resistance afforded by the various components of your equipment. Once you can do this, you should be able to freebuzz a tone or series of tones without the aid of the mouthpiece or instrument.
So to reiterate, once you can form an embouchure without any equipment, you can practice and strengthen those muscles without buzzing at all.
Some other notes:
- Lip whistling (as opposed to palatal whistling or tooth whistling) uses similar, but not exactly the same muscle groups. This is a bit more socially acceptable than freebuzzing while walking around in the park. :-)
- Warburton Brass has a tool called the "P.E.T.E.", or "Personal Embouchure Training Exerciser", that is ostensibly designed to help you strengthen your embouchure by giving you a center point to focus your lips around and creating resistance for your lips to pull against. Plenty of marketing and usage material can be found at their website linked above.
- Another (much less expensive) object to focus your embouchure around is a pencil (not the pointy end) or similar rod-shaped object. The idea is to keep the rod pointed forward in the direction of where your airflow would be -- using gravity as resistance for your embouchure.
Any kind of training tool like those mentioned above must be used carefully, so that it is being used to strengthen your embouchure, not that your embouchure is being trained to fight against the tool. Of course, all methods above must be used in balance with plenty of practice with the instrument. You are NOT going to be able to compensate from hours missed on the horn with "strength training" away from it. Pace yourself, set achievable goals, and reliably track your progress.
I'm going to start as if you have already played a brass instrument and have some knowledge of them. As a former trumpet player and teacher I found that if you copy the same position and feeling w/o a mouthpiece as with a mouthpiece you can get great benefit. Trombone or trumpet.
So practice the same embouchure w/o the mouthpiece. Form the same shape w/ the mouth and lips. Blow the air thru the lips allowing them to vibrate "buzz" the same way.
Don't forget to breathe w/ your diaphragm the same way also. Proper breathing is half the battle in playing. It can make it easier on your lips.
Doing everything the same but w/o the mouthpiece will give you a workout routine plus a practice routine. You can, in a pinch, play music this way. Play w/ the radio or mp3. Jam w/ your friends.
Also, alternating between "buzzing" and whistling will help to strengthen your embouchure. Kind of like push ups.
One last thing is kissing (not kidding) will give you a workout and is also satisfying in a number of ways.
I've used them all. Combine them. Pick and choose. Play when you practice and practice when you play. Except when you're being paid, then play when you play.
If it stops being fun choose a different instrument.
Always have fun.