When playing the 4th fret string D there is a buzzing sound. You can't hear the buzzing when the bass is plugged into my amp, only when the bass is alone. Should I be worried about this? Any solutions?
In general, it's fairly common for a bass set up with a reasonably low action to have a few rattles and buzzes here and there and not something to worry about.
However, if it's bothering you at all, its worth seeing if it can be solved without compromising on other aspects of the instrument's playability. If the rattle is coming from the string hitting the frets, it probably will be affecting the sustain of that note.
If the rattle is coming from the string hitting the frets, solving this might in general be done in one or more of 4 ways : adjusting the action at the nut, adjusting the action at the bridge, changing the amount of relief in the neck, and making sure the frets are levelled properly.
If you're only getting a rattle on the 4th fret D string, I'd start by checking that the 5th fret isn't a little high. This might be something that you need someone more experienced to show you how to do (and fixing it will need the right tools and knowledge too).
It's likely you can make the problem go away by raising the action at the bridge on the D string saddle, but if you only have a problem with one fret, that will be giving you an unnecessarily high action.
Sometimes you can get 'sympathetic' rattles coming from a loose screw or stray bit of unclipped wire on the headstock too, so listen carefully and see if you can track down exactly where it's coming from.
When I play my fretted bass without amplification, then the buzzing and clattering is in fact much louder than the actual notes!
As you say, this kind of noise is generally much less present in the amplifier signal, which is after all the important thing which the audience gets to hear. IMO, the buzzing that you do hear over the amp actually tends to have a positive effect: this adds to the “growl” of the bass, and gets you heard in the mix and adds definition to the notes.
Now, all that is assuming a somewhat uniform, controllable buzzing, the kind that's caused by the strings clashing with the fretboard. This should largely vanish when playing quieter (again a useful thing, as it means you can achieve a dynamic sound even when the signal is strongly compressed). If only a single note is buzzing, and even at quiet playing, then this is at any rate not musically useful. It might be a good idea to let a luthier check what's going on.
But quite possibly it simply doesn't matter. If this is some kind of sympathetic resonance at the headstock, then it probably won't be audible at all in a band context, and should anyway be exclipsed by fretboard buzzing if playing at typical rock volume.