On and off beats really exist on a spectrum, and determining whether beats 2 and 4 are on or off is really a question of tempo.
If the music is fast enough, we can start to feel the 4/4 time in (as we say) a large 2, meaning that
1234 starts to sound like
1&2&. In such cases, 2 and 4 are definitely heard as off the beat, with 1 and 3 being on.
But if the music is slow enough, 2 and 4 are so far away from their surrounding beats that we really perceive them as being beats in their own right; as such, we then view
1234 as all being on the beat, and it's the material in between those beats (the
&s, for instance) that are viewed as off the beat.
This is one reason why using the strong/weak terminology can be more helpful: beat 2 will (basically) always be viewed as weaker than beat 1, but sometimes beat 2 is on the beat and sometimes it's off. In other words, the on/off dichotomy is often too black and white, but the strong/weak spectrum is much more malleable.