If we take drums for example like snare, kick, cymbals, etc. They don't have a pitch I was wondering if they're in the "noise family" like pink, white, brown noise. Generally, is anything that doesn't have a pitch considered noise?
I think when an instrument doesn't have a clear pitch, the musician's go-to word is 'unpitched', rather than 'noise'. Noise in a musical context often refers to something extraneous or unwanted - e.g. when talking about fret noise, or a noisy amp.
However, you are quite right that the sound of many drums is essentially very like 'enveloped noise' - and from a synthesist's point of view, that's often how you might achieve an imitation of the sound - take filtered noise and apply an envelope to it. Depending on how the noise is filtered, this technique can create sounds from bass drums, through snares, to cymbals.
Digging into things a bit further, there isn't necessarily a very clear distinction between 'pitched' and 'unpitched' instruments. As Tim says in his comment, some drums are tuned to a pitch. Equally, pitched instruments like flutes often have a 'noisy' aspect to their sound.
Even from the synthesist's point of view, you could create a 'noisy' sound by mixing a load of sine waves at different (though still specific) pitches, or you could make a pitched sound by putting a very sharp filter on a noise source. There isn't really a strong distinction between a the actuality of a 'pitched' sound, and very narrow-band noise.
As so often, there's no tidy yes/no answer to your question, but it's interesting to consider the various ways a musician might use the term 'noise'.
'Noise' might mean random sound. Or a definitely UN-random mix of frequencies such White Noise or Pink Noise. Or a randomisation of digital values applied as 'dithering'. Or simply unwanted sound.
There's more pitch in most drums than you might imagine! And why do you think we use large AND small cymbals? Let's look at some better examples. The initial transient of a plucked string perhaps, or the mechanical sound of a piano or flute mechanism. The pluck transient contains un-pitched sound, but it's wanted. Call it noise if you like. The key clatter is more clearly UNWANTED noise. Dithering noise is an enigma - deliberate noise which has the effect of REDUCING perceived noise!