I cannot figure out the actual difference between these two beats.
Isn't it sufficient to have only 2/4 (or 4/4 )?
Go for a walk. Count each step, in twos or fours. Tread heavier on the ones.
One two One two One two One two
One two three four One two three four
They feel different, don't they? This is the difference.
And yet there is an equivalence between them. Walk at the same tempo, but count to four twice as fast, so you're stepping on the One and the three.
One (two) three (four)
This feels like the 2/4 rhythm again, except for the in-between-steps counting you're doing.
This sort of equivalence comes up all the time. In environments where musicians play without sheet music, it's quite possible that the drummer thinks he's playing in 4/4 (with triplets), while the guitarist thinks he's playing in 12/8. In this example, what the drummer calls a triplet, is three 8th notes for the guitarist.
When a composer scores their music, they choose the time signature that they believe conveys the feeling most clearly to the musicians, while being easy to read.
The other answers are all essentially correct, but I think a critical point is missing.
There aren't just fully "strong" and fully "weak" beats; there are also beats of medium strength (and other varieties). 4/4 is emphasized like this:
Note the half-accent on the third beat, different from what slim mentioned. If you're playing triplets, that works out to:
Slim was right in saying that 4/4 with tripletted quarters indistinguishable from 12/8 with normal quarters, but it's only true because of that emphasis on the third beat. 12/8 is emphasized like so:
With quarters (combining pairs of eighths) that works out to:
Which is exactly the same as the above 4/4 with triplets.
Now to get to your specific question, 2/4 has no medium beat, so four quarters would be emphasized like this:
This emphasis gives a noticeably different feel to the music than 4/4 does.
It also has to do with strong and weak beats. In 2/4, everyother beat (the one in each measure) is a strong down beat. In 4/4, every fourth beat (the one in each measure) is a strong down beat. It is all about accents and feel, so there is a major difference.
It's all about the feel in the music. Sometimes it really doesn't matter, and it is very difficult to say what is more natural.
But often you have distinguised beat at the first in each measure.
If it is a march, you have the first beat on the right foot when marching to it, and it is natural to have a 2/4.
Rock music often has a beat pattern that more or less repeats itself each 4th beat, and thus it is more natural to have a 4/4.