I think the basic answer is that the vast majority of classical music used notes beyond the diatonic scale, and even in pop music today it's still pretty frequent to have chromatic notes.
Thus, if you're playing in the key of D major, and there's a C-natural, you need to write C♮ with the natural sign. If this continues for several measures (as sometimes happens with a temporary modulation), it's helpful to have a reminder at the beginning of each staff system that says, "Oh, yeah, C♯ is still the default." The more chromatic music gets, the harder it can become to remember what the "default" sharps and flats are in the key signature, so having a periodic reminder is helpful.
Also, even if you're in one key, the notation can be harder to understand for music beginners, who may not make immediate connections between the sharps and flats notated on each line and the key of a piece. This is particularly problematic in minor, where the 6th and 7th scale degrees also tend to move about and can contain various accidentals. Beginners in music are often taught about "natural minor" and "melodic minor" and "harmonic minor," but actual minor key music tends to use a lot of accidentals on those 6th and 7th degrees. Again, having a visual reminder on each staff of the accepted defaults can be helpful.
And it's often visually more helpful to see those sharps or flats on the lines and spaces where they actually occur (and can sometimes be cancelled with naturals or added to with additional accidentals). In fact, it was still accepted practice at times even in the 18th century (and later) to include those key signature sharps and flats on every place in the staff that they'd occur -- for example, a piece in D major in treble clef might mark the F♯ both on the top line of the staff and in the bottom space. In that sense, modern notation has simplified key signature notation a bit.
(Just as a historical note, part of the reason for that previous practice was because clefs were not particularly fixed in their staff positions until some point in the seventeenth century. It was quite common for clefs to move up or down by a line or more, even multiple times in the same piece. Performers reading a score needed to be able to immediately reorient themselves on every line of a score to where the clef was and therefore where the notes were. So having default sharps and flats marked on each staff line or space was also helpful in orienting to each staff line. We've dispensed with some of the other notational practices needed back then, such as the custos, a symbol that often looked a bit like a checkmark and appeared at the end of each staff system to tell the performer which note was the first note at the beginning of the next system. Again, this was needed at a time when clefs could move about freely, and performers needed to immediately know how the notes from one line of music connected to those on the next line.)
All of that said, modern practice is inconsistent. Sometimes things like fake book lead sheets will dispense with the practice of writing the key signature on every line, merely giving it once at the beginning. (Often they also omit the clef except for the first line.)
It really all depends on how much consistency you have. In a mostly diatonic piece of music that remains in one key, I agree that one could easily just write "D major" on the top line and everything should be clear. However, the more chromaticism, key changes, etc. that occur, the more it is helpful to have visual reminders of what the current default sharps and flats are.
On the other extreme, many 20th century composers who began to write highly chromatic and atonal music sometimes dispensed with key signatures altogether, instead preferring merely to note all accidentals when they occur. If you really wanted to simplify music notation, that practice would actually be most helpful, I think -- any note without an accidental is assumed to be natural, and when sharps or flats occur, they must be written before every note. That would make music notation more accessible to beginners and be consistent across styles of music that are diatonic, highly chromatic, or even atonal. However, that practice would require even more notated sharps and flats than our current practice of putting key signatures on every system, so it's unlikely to be adopted anytime soon.