The moonlight sonata 1st movement by Beethoven, for example, has a melody on top with an accompaniment with the left AND right hand (starts at measure 5). People usually say that you play the melody with your pinky louder than the accompaniment. But why? In the sheet music, that has not been specified, but if you were to specify it, you would have two dynamic markings for one staff (accompaniment right hand soft, and melody right hand loud). Is this always the case, that you play the melody louder than the accompaniment, unless specified?
Yes, you play the melody louder than the accompaniment, unless specified.
Questions regarding the relationship between melody and accompaniment are easily answered, if one thinks about, what melody and accompaniment are in reality, namely different voices. In the case of the moonlight sonata we have most of the time 3 voices: Melody, Middle voice (the running triplets), and base.
Think about the piece played by an orchestra. The Middle voice could be the strings, the base will be cello and contrabase, the upper voice could be some wood-instruments.
Now it is immediate clear, that the strings will not play louder then the wood, because this would sound very strange.
The dynamic marking in the scores thus tells us the loudness of all instruments played together, not the dynamic of each instrument.
This is not generally easy to answer since "louder" is ill-defined anyway: on a piano, the lower notes have more string mass behind them, and an accompaniment more often than not has a whole number of notes being held in contrast to a single treble line.
Added to that, the piano is a percussive instrument. Historic predecessors like the cembalo (at least single-manual variants) were not even capable of producing different loudness for melody and accompaniment.
So the important thing is that the melody can be discerned rather than it being stronger in volume. A whole lot is already accomplished by it being usually on top, making at least its overtones occupy a comparatively uncontested frequency space. A lot can be achieved using articulation: this is a tool that is a bit underrepresented in the piano player's conscious toolbox since the release is somewhat underrepresented in importance compared to the attack, however it still is important for creating coherent melodic lines that stick together in defiance of accompaniment.
Then for orchestration, one can give the melodic lines signature sounds, instruments that don't have all that many overtones but rather particular ones, like an oboe. Those tend to be comparatively robust to follow even against a not-all-that-quiet orchestral backdrop.
Basically human hearing has a lot of features it is able to lock onto, and loudness is one of the more reliable and less subtle ones. But relying on it solely gets old eventually. It's good if you keep other tools in your box.
By convention, usually yes. By necessity, no.
Music written as melody/accompaniment is generally intended to follow that approach and is usually performed that way.
There are (of course) exceptions. In much Baroque counterpoint the listener can focus on one or more voices at a time or can listen to it as a whole.
Also, there's no rule against a composer moving melodies from the background to take center stage, or moving a prominent melody into the backdrop.
Many Classical pieces do this de facto by presenting accompaniment (which can be perceived as melody) before the intended melody begins. Moonlight Sonata, in fact, has this trait.