I want to start playing melodies and chords across the whole neck and escape the prison that is the first 4-5 frets, but I don't know a systematic way of learning how to do that. I thought of learning a ton of scales, modes and chord inversions but that seems like a lot of memorisation which I could do, but I don't know if that's the way to go. Could anyone point me to the right direction?
Chord-wise, there's the CAGED system, or in my world, CAGE system, as the D shape doesn't lend itself as fully as the others. It uses the basic open chord shapes of, surprise, C, A, G E (and D) which can all be moved about the fingerboard using a barre behind, where the nut would be. They produce different voicings of chords, so, for example, open C= A-shaped C on 3rd fret, E-shaped C on 8th fret, G-shaped C on 5th fret and at a push, D-shaped C on 10th fret.
As far as scales go, they again are a moveable feast. Learn a two octave major or minor scale, across all six strings, and learn where to plant your hand (on which fret-span) for the main keys. I say that, because if you know where the D area is, C# is only one fret lower, E♭ one fret higher. Easier to remember seven rather than twelve!
The same concept occurs with other scales - blues, pents, etc. They all relate to the caged system in location, so it's not too bad to map out patterns. You'll find that the lower notes (on the board) of one shape are the very same notes as the upper ones on the next shape down, so soon you'll join it all up.
The way I do it is: for each melody note + chord combination, (1) find a suitable chord fingering with the bass inversion you want, such that you can reach the melody note on the top two or three strings without leaving out the most important notes of the chrod (bass note and third i.e. is it major or minor), (2) locate the melody note somewhere on the top two or three strings.
Simple as that! At first you will be very slow, but you'll gain speed and efficiency. Just keep doing this for more and more songs, and you'll start noticing re-occurring fingerings. Plausible, sensible pop songs rarely need impossible fingerings.
My advice is to try and think beyond the guitar neck. You can learn the underlying patterns of the major scale (and by extension all the notes) as it appears on the guitar neck (or any instrument tuned in fourths). As an experiment, tune your top two strings up by a semitone, so that they're c and f (now the guitar is tuned in fourths all the way), and get some grid paper. Pretend that each square in the grid paper represents one note on the fretboard. Now with your guitar, find as many notes as you can from a major scale, say C major, and colour in squares on the paper to match the pattern you are uncovering with your guitar. Don't worry about the names of the notes, just focus on the pattern. Once you've covered a fair portion of the paper, study it for patterns. These patterns can be used to know a note on sight without actually knowing exactly where you are on the guitar, much like how on a piano, the way the black keys are grouped allow you to tell what each note is. For example, when you draw out the grid, one pattern you'll see are 2x2 boxes that go diagonally upwards corner to corner. The bottom right corner of any box will always be C. Knowing the patterns will also unlock many many tools later, like knowing all your modes without memorization and being able to build chords from scratch. Hope this wasn't too confusing lol