If you're playing straight and switch to playing swing, the 'and's move but the strong beats don't.
It might be that the players who are trying to play swung are pushing the strong beats around too, which would be disconcerting to the other players.
I'd suggest an exercise where everyone plays the same line (a fragment of a scale for instance) first straight and then again swung. You can also try clapping exercises like this, where you silently count the lower-case and then CLAP the upper case:
one-and two-and three-and four-CLAP
The 'ands' and the CLAP move around depending on whether your quavers are swung or not.
The other thing I tell my beginner brass band when they're playing swung quavers is to make the first note in each pair long, and to make gaps between all notes as short as they can, whilst still articulating every note and not slurring it. Without this reminder, the tendency is to play the first note of each pair too short - and the result is a very 'polite' performance. Once you lengthen the first note in each pair the performance has far more attitude and sounds much jazzier.