I know that every major and minor scale can be thought of as the notes of a C diatonic with specific sharp and flat modifications. This way of thinking works well for simple keys like G, D, A, F, etc. but for me it kind of falls apart with keys that really far from C on the circle of fourths, like Db, Gb, etc. For those I just end up memorizing which notes are in the scales and thinking of the chromatic scale relative to the key sort of as a group of 12 equally important notes. This also makes it easier because it avoids the whole switch at like C# major to Gb major and having to change back and forth from sharps to flats.
Is there any reason not to think like this? Do the notes of C diatonic always have to have special preference in our playing? I'm specifically asking from a jazz perspective, but all thoughts are welcome.