In the context of Baroque dance music or suites, then there are good reasons to use 3/8 in preference of 3/4 (or vice versa). In the days before metronomes, how the music was notated would be an indication of performance speed. The notation would also be specific to a particular dance.
I have borrowed diagrams from Jan van Biezen, who has written extensively on such subjects:
The above image (figure 1) is fairly self-explanatory (the metronome marks are a modern interpretation of a 1705 French work by l'Affillard, I believe): faster dances with quicker steps are notated in 3/8, slower ones in 3/2 or 3. N.b. it appears that 3 can mean duple 3-time similar to 3/2 as well as what we would recognise as modern 3/4 time.
Now, the following diagram (figure 2) summarises the transitions between using different metres and tempi. Van Biezen writes in Dutch; Geheel means whole as in normal time where you are really concentrating on counting in 3, think of a Menuet (3/4), and Half means half as in half-time, like a Sarabande with a drawn-out beat but with possible subdivision of that beat (3/2).
Figure 2 is saying the following: if your dance/music has a 'whole beat' feel to it, then use one of the time signatures on the left. If it is fast, use 3/8, if it is slower, use 3/4. If your dance has a 'duple subdivision' feel, use a time signature on the right. If it is fast, use 3, if it is slow, use 3/2.
However as time progressed, it seems the antiquated 3 was replaced by 3/4, which is a sort of one-size-fits-all time signature in that it can accommodate both 'whole' and 'half' time feel.
Take four cases (I'm using Bach cello suites as examples because they are based on dances as was the done thing back then, and also because they're awesome pieces of music).
- A grave Sarabande in 3/2, (bottom right of figure 2);
this is slow and the duple subdivision is accentuated in some bars,
especially at cadences and throughout the second part of the piece.
- A slightly more urgent Sarabande could be written in
l'Affillard's antiquated 3 (top right of figure 2) to show that
the beat should be between crotchet = 60 and 120, but that you still
have the duple subdivision. Bach, however, writes it in 3/4,
which is acceptable and we still understand that it's a sarabande.
- A menuet would have a gai tempo, but the metre is somewhat
ambiguous. Obviously there is stress on each beat of the bar, but
we could place emphasis on the quavers too, as in this example. We
might therefore use the antiquated 3 signature (top right of
figure 2), but really 3/4 (bottom left of figure 2) is adequate
and this is what Bach uses.
- A vite gigue is unambiguous, Bach writes it in 3/8
because it's rapid and we feel the whole beats (top left of figure
Later on his page, van Biezen also refers to the time signature as determining the shortest value at which notes inégales can be used. It's not strictly important here but is quite interesting, so read the pages linked in this and above paragraphs.
Furthermore, as has been alluded to in other answers, in both Baroque dance music and more modern pieces, 3/8 can be used as the sub-unit of a hypermetre (i.e. counting one-in-a-bar to express a phrase lasting longer than 3 quaver beats). This may be useful if the composer wishes to highlight a sense of urgency in the music, or perhaps if it becomes somehow easier to represent complex cross- and counter-rhythms.
This might be interesting reading too, a discussion on whether Bach really did use time signatures as tempo directions.
Tl;dr: If you're in 18th century Europe and need your crew to know how fast and in what mood to play your new triple-time dance piece, you would choose 3/8 or 3/2 etc. as a matter of convention.
If you're in 21st century anywhere, use the time signature that you think conveys the mood of your piece or which you feel best expresses your phrasing.