There are some good answers here, but I would like to answer two of your points in a way that nobody here has used.
It seems very redundant to have both sharps and flats (not to mention double flats and double sharps)
At the first glance it is so. However by discarding the other accidentals you rob yourself of a lot of features that make reading scores considerably easier.
Here's a C major scale:

Notice how the dots are nicely lined up. Each dot is one line or space above the previous one, and each line and space is occupied by exactly one dot (within the scale). That makes the scales very easy to spot. I also feel that it's very natural to represent scales like this.
Now consider F major. Traditionally, you write it as shown on the left. Since each degree of the scale has its own line/space, you can introduce key signatures that can target each note separately. So you can write the scale in the way shown on the right too:

If you forbid the usage of flats, suddenly it's very hard to do this. You could write an A# instead of the Bb, but that will put two dots on the same space and the next line will be empty, so the nice properties are lost. The only way to keep the nice properties and write the scale without using flats is this:

I certainly prefer the traditional way.
In fact, there are lots of similar features in the traditional notation. There are quite some patterns that make reading simpler: for instance, if you're in A minor, the dominant chord is E major, written as E-G#-B. Now you use the same pattern in the other keys too, so in B flat minor the dominant chord is F-A♮-C (you had too many flats, so instead of a sharp you use a natural), and in G# minor you would use D# major, written as D#-F𝄪-A#. Each time you used a different accidental for the middle note, but it is always "one semitone sharper than the rest of the key". (By the way, the reason for using the double accidentals is just upholding these patterns even in keys with lots of sharps or flats.) If you forbid using some accidentals, this breaks for some keys. (Also the chords would need to change their "shape" on the staff in some of the keys which would make them harder to read.)
Here's an image to make it more clear, hopefully:

In the first bar, there is a very simple chord progression in A minor. In the second bar, I have written the same progression, but transposed to D♯ minor. You see that if I use a double sharp, it looks just like the original. However if I forbid using double sharps, I need to write what is in the third bar. You can certainly see that the chord highlighted in red now looks different (it's no longer a nice stack of three notes), even though it's the same chord, so in this way we have only made it more confusing. To get rid of this confusion, we use double sharps. (Similarly for double flats in other situations.)
Is there an advantage to memorizing unpleasant things like the circle of 5ths instead of just doing mod 12 arithmetic?
Yes. There is a decisive advantage. Suppose you have two different major keys. Now let's define the distance d(A,B) of those two keys as the number of notes in which they differ (not taking the enharmonic equivalents into the account, so for the purpose of this definition, A# = B♭ etc.)
For instance, C major has the notes C, D, E, F, G, A and B, and D major has notes D, E, F#, G, A, B, C#. They share 5 notes and differ in two, so d(C major, D major) = 2. However, the C# major scale has the notes C#, D#, E#, F#, G#, A#, B#, so it share two notes with C major (E#/F and B#/C) and d(C major, C# major) = 5.
I think that this notion of distance is quite natural. (This is very useful. For instance if you are in a certain key, you want to harmonize melodies mostly using the "nearby" chords in this sense.)
And now the important thing: on the circle of fifths, the adjacent keys have always d = 1. So d(A, B) = the number of steps you need to take on the circle of fifths to get from A to B (taking the shorter way). I think that this makes the circle immediately useful and well worth remembering. (And by the way, the circle measures the distance for the minor keys in just the same way.)