The idea isn't necessarily to memorize all chords and scales, per se, but to understand them and make them second nature. In reality, becoming more aware of the patterns within each key and of each chord type will essentially force it into your memory. What you don't want to do is write out all of the notes of a given scale or chord type and read them back to yourself to try to memorize them. You want the memorization to be the result of playing and seeking to better understand the patterns behind them. You should find that there are a few patterns that become very obvious and you can play the correct chord without thinking about it.
You can try taking each interval from a chord type and playing that through all of the keys. This should familiarize yourself with the shapes necessary to play those chords. I would suggest starting with the perfect fifth (P5), since that is the most common interval between different chord types, specifically the most commonly used chords, major and minor. There are a couple ways to approach this and I would suggest using as many as possible. Perhaps start by playing a P5 in each hand, then moving it up through the octave, for example, play C and G, move up to C#/Db and G#/Ab, onto D and A, etc. The pattern that you should see is that in most instances, if you have a white note as the root, your fifth will also be a white note, and same for the black notes. The exception here is for Bb/F and B/F#. I would then try to work through other patterns, such as moving through the circle of fifths (C, G, D, A, E, etc., or C, F, Bb, Eb, etc.). This will allow you to become familiar with the shape of the fifth from each root when moving larger distances than a half step. I would then take this concept and apply it to thirds (major and minor separately). For instance, C and E, Db and F, D and F#, etc., then onto circle of fifths (C and E, G and B, D and F#, A and C #). As you progress as a musician, you should find this all coming more naturally and as you move onto more difficult/complicated chords, these patterns will be extensions of your previous knowledge and emerge more quickly.
As for scales, you will want to practice all of them and make sure you're using appropriate fingerings while doing so. This is a monotonous process for most people but it's pretty necessary to accomplish solid technique. Proper fingering encourages your hands to become more familiar with how to move from position to position smoothly, which will allow you to play more difficult parts, particularly faster lines, more easily. In this exercise, I would also encourage you to pay attention to the patterns within the different scales and different fingerings. You should find that there are some fingerings that are the same from key to key, such as C, D, G, A, E, and B.
Due to the boring nature of practicing in this way, it is a very good idea to have actual pieces of music you are playing as well. Try to notice the patterns within each piece of music as well. As you play the melody, you can notice how the scale patterns that you've practiced appear and how the fingerings are often the same but also that they are sometimes a little different based on the line in question.
The end goal is to know all of this stuff but to just know it instantly, not having to think it out. The way you choose to practice can encourage this. The important thing is to make sure that you find a way to make this enjoyable for yourself, especially if you're not the kind of person that will force themselves to do something they don't like. If you can't find a way to make these exercises enjoyable, then I'd recommend finding a way to force yourself to do it at least a little bit at a time. Choose 2 or 3 scales for that day's practice session, then go through each of them a few times for maybe 10 minutes, go play some music you enjoy, then come back to the boring stuff for another 10 minutes, and go back and forth throughout your practice session. The important part is that you are consistently practicing everything so that it actually gets ingrained. If you just do a couple keys once a week for a minute or two, it's not really going to stick; you really want this to be a daily exercise. You also want to make sure that you're covering all of your keys over time, so it's a good idea to keep a practice journal, where you can keep track of what you've done in previous sessions to make sure that you're not neglecting some. If you find that certain things are coming more easily than others, it's a good idea to try to more frequently practice the things that are more difficult but don't just stop the stuff that is easier.
This may not be the best regiment for you as an individual, since everyone learns differently, but it's the best general advice I can offer without knowing how you learn. I would ultimately recommend getting a private instructor because they will be able to cater the lessons and practice regiment to you specifically. The other important thing that a teacher can provide is more of the technique. There are tendencies that are unhealthy or inefficient that if practiced for a long time, will be incredibly difficult to reverse. Learning the correct technique early on will save you a lot of trouble, and potentially physical pain, later on.