I heard that Mozart heard Gregorio Allegri's Misere Mei Deus once at a concert and copied it down note for note from his head. I can copy down a piece note for note if I have it in my head, but I'm really frustrated because I'm having trouble listening to more than three or four voices at a time when listening to dense polyphonic music (5-15 voices). Is still enjoy listening it to an extent, but it feels increasingly more like a reminder of my ear's incompetence than anything else. How did composers in the Renaissance listen to 8 simultaneous, independent voices? I'm wondering if there's anything that I can do to help me improve this skill.
This type of instantaneous recognition and notation is really something only possible for individuals with absolute ("perfect") pitch. And acquiring that is typically only possible for young children that are surrounded my music and musical instruction from such an early age (like Mozart was) that these skills are as natural to or I speaking, hearing, and writing in our native languages.
So don't fret if you can't do this. I have a PhD in music theory, and with a single hearing I could probably get, at best, only eight measures or so of only the highest and lowest voices. (Note that I don't have absolute pitch.)
With that said, there are common stylistic patterns that composers use that could help you with this. The opening chord progression of Mozart 40 is the exact same as the opening chord progression of the opening prelude of Bach's Well-Tempered Clavier (only the Mozart is in minor), and recognizing this can quickly help you fill in outer voices even if they aren't literally what was written in the score.
A similar question could be how these composers conceptualized writing these 8 voices (not to mention 40!), which may be more in line with what you were thinking.