As a saxophone player (it's a monophonic instrument, so I am largely unaffected by chords, unless I am playing multiphonics or "virtual" chords in my head during solos), I found it slightly easier to play key signatures that have flats rather than sharps when I was a beginner, because of the mechanics of my instrument. To give you an example (pictures taken from The Woodwind Fingering Guide, I flipped them for your convenience, so that you can see which keys are higher and which are lower on the instrument when it is played; blacked-out keys on the diagram represent those that are pressed):
A4: 
A♭4/G#4: 
G4: 
Explaining the example... If I am moving from A4 (a higher note in terms of sound) to A♭4 (a lower one), I have to press 2 more keys below A4, so my downward motion reflects the equivalent motion in sound. If I am going from G4 (a lower note) to G#4 (a higher one), I have to counterintuitively press 1 more key below G4, contrary to the motion in sound.
There are, naturally, some exceptions to this:
- the more "unusual" notes like C♭4 (enharmonic of B4, upward motion on the instrument) and E#4 (enharmonic of F4, upward motion)
- low-end notes requiring the left little finger (B♭3, B3 and C#3; the left hand is above the right on the saxophone, but it doesn't feel as much of an upward motion, because it's handled by the little finger when all the fingers on the right hand are already down, which makes it feel more like a "cherry on top" than an actual proper movement)
- some upper-register and altissimo notes
- some of the more unorthodox trill, alternative and microtonal fingerings
But, overall, as a saxophonist you quickly learn that playing a flat means going down the instrument, towards the bell, and playing sharps involves (at least for me) some mental gymnastics (they become less and less problematic with time) that translate sharps into flats. I still find it easier to play scales descending and play the occasional D4 instead of an F4 when I see an E#4 and am very tired/drunk. Your mileage may vary though.
I think that might be the case for many instruments that do not use a linear structure (like the piano) or a matrix (like guitar) to represent notes. To all oboe players out there: I can feel your pain.
P.S. My personal "favourite" on the saxophone is D#4 → E#4, which involves lowering/lifting 3 fingers to go down/up 1 tone, 1 "unnecessary" accidental (F natural is so much nicer than E#) and a note (D#) which likes to sound muffled/split in the 2nd octave of the instrument's "normal" range. If you are exhausted and trying to translate sharps into flats while playing a fast passage involving this switch, you can miss the timing or play something else note-wise entirely (if this switch is for the 3rd octave of the instrument's "normal" range, it becomes even more problematic and is considered one of the most difficult ones for the instrument, so you really don't want to play it with "unnecessary" sharps).
Update: this information is based on my personal experiences of playing "regular" B♭ and E♭ saxophones. There are horns out there that feature a linear fingering design.