While many things said in these answers are correct, I think the actual difference has been missed.
The most important thing to remember when considering the difference between the two is probably that people often speak of 3/8 as having 'three beats in a bar' - but it actually has one.
3/8 has one beat in a bar, 6/8 has two, 9/8 has three, and 12/8 has four, where each beat is a dotted crochet.
There is indeed a difference in the notation of the two, but the main difference is how they feel when you subdivide them (divide them into shorter note lengths such as when counting in your head).
Subdivision is very important for orchestral musicians as it allows them to keep in time and understand how to play rhythms which look strange at first sight when sight reading, for example. Conductors will often tell you about their 'internal metronome' - many constantly hear the subdivision of the beat in their heads while conducting.
So what's the difference?
In 3/8 time, there is one beat in each bar (one dotted crochet) and each beat feels like it's in three - you can split each beat up into three quavers.
In 3/4 time, there are three beats in each bar, and each beat feels like it's in two - you can split each beat up into two quavers.
So, when you listen to a piece in 3/8, you can clap to the beat and, depending on the speed, you can count 'one, two, three, one two three...'.
But when you listen to a piece in 3/4, you can clap to the beat and, as you clap, say 'one two, two two, three two, one two, two two....'
So in conclusion, a 3/8 piece will sound like it's in three, but a 3/4 piece will sound like it's in two, even though there are three beats in a bar.
So whereas it can be very difficult (or not really possible) to tell the difference between 4/4 and 2/4 when listening to a piece, and both are often considered correct in aural exams when it is hard to tell, there is a very notable difference between 3/4 and 3/8, and the two do not actually sound the same.
Hope this helps!