There are a couple of factors here.
Some instruments are similar, but vary in terms of their technical difficulty
For example, if you were to compare the flute, clarinet, oboe, and bassoon, the flute has the least amount of variation between registers, required alternate fingerings, and keys in general. The clarinet has more keys, different fingerings across registers, and slightly more resistance on articulation. The oboe requires regular use of half-holes and unintuitive fingerings and has a much higher pitch variability. The bassoon has the most complex fingering system, high pitch visibility, and more physically difficult to perform fingering shifts.
While all four of these instruments are capable of performing similar feats of dexterity, in general it is slightly easier to play very fast, technical pieces on the flute, and also slightly easy to perform acts of transposition because of register symmetry.
Some instruments have difficulty tiers based on entirely different mechanics
For a very obvious example, consider a flute vs. a piano. Some of the challenges in flute (tone production, articulation, intonation) don't even exist when playing piano, while conversely the flute player needn't worry about multiple concurrent moving lines, pedaling, or consistent key pressure.
In other cases, players control similar factors using different mechanisms. Compare, for instance, the flute vs. the violin.
Tone production:
- Flute: Breath support, embouchure, tongue position
- Violin: Bow, controlled by right hand and arm
Intonation:
- Flute: Angle of air, embouchure, length of instrument
- Violin: Left
hand finger position on string, string tension
Articulation:
- Flute: Tongue, breath control
- Violin: Bow pressure, angle of hair, bowing speed
Note Selection:
- Flute: both hands work together to play one note
- Violin: one or more fingers work together with the bow to play one or more notes
As you can see, all these mechanisms are very fundamentally different, so the kinds of things that are "hard" on flute might not be the kinds of things that are "hard" on violin, or on brass instruments.
Some players are just not often presented with challenging parts
For a mixture of legitimate acoustic reasons and less legitimate ignorance reasons, many composers don't write very demanding technical parts for certain sections. Your tuba and euphonium parts are far more likely to play stable, consistent rhythms in a more narrow range, while higher instruments are more likely to get moving, technical parts because it's easier to hear moving parts on the top of a chord, and also because that's what they're most familiar with. Yes, it's kind of circular.
That doesn't mean that they are incapable of doing these things, or even that their instruments are poorly suited for them. It just means they are less likely to have experience with them if they're not practicing outside of rehearsal setting.
Conclusion
While there are differences in technique across instrument sections, none of these are excuses for playing in different key signatures or counting complex rhythms.
Modern wind instruments, including brass instruments, are fully capable of playing in all keys. String instruments are also fully capable of playing in all keys. While there are some keys that are "easier" than others for most instruments, if anything it's actually less work to play in distant keys on brass and strings than on woodwinds. All the ratios between notes are the same, regardless of what key you're playing in, and the intervals between those notes are much more apparently reflected in the fingerings for brass and string players than for woodwind players.
Yes, it's harder to jump around and tongue fast on brass, but it doesn't take very much searching to find virtuosi playing Flight of the Bumblebee on just about any instrument. This is not to say everyone needs to be able to do that, but key signature complaints are just them being lazy. Go practice your scales, learn your pedal tone fingerings, and work on some counting, low brass!