There is no blanket answer to your question. It depends. "It depends" isn't very useful though, so let's try to dive a little more into it.
Advantages and disadvantages
Sample: Less complexity, less flexibility.
Synthesis: More complexity, more flexibility.
But it's not that simple, and the weight of those cons and pros depend on each specific scenario. Do you want realistic cymbals? It can be very complex to do that through synthesis, and the result is often not "real enough", but through sampling you can do it very easily (you can end up liking the less-realistic synthesized cymbals more though, who knows). Do you want a sawtooth with even harmonics panned opposite to the odd harmonics? (like here) How do you even achieve that through sampling? You go the synthesis route.
Many synthesis techniques involve sampling, so that generality becomes harder to hold. I'm assuming with sampling you are referring to simple samplers.
When should I use one over another? Should I use recorded samples or create synthetic sounds?
It depends on the limits of the technique, preference, and your skill set.
Are you experimenting? If so, it's a good idea to use both (and others) and hear what you like the most, maybe using both at the same time. When I'm designing kicks sometimes I like to mix layers of drum-kick samples and fast-frequency-swipe type bass drums (like in the Roland's 808).
Do you already have something in mind? First ask yourself if what you have in mind is achievable with both methods. If so, ask yourself which method will have the best impact in the sound. This is not exclusive to sample vs synth, but also useful when deciding among synthesis types (FM, AM, granular, additive, whatever). If you are not sure of which one is better suited for the job, try both and hear which you like the most.
In short, it comes down to experience
The answer to your question will vary depending on the sound, on each individual case. Only experience (or a less general question) can answer it consistently and accurately. Once you study and understand the different synthesis techniques (sampling, subtractive, additive, FM, AM, pulse modulation, phase modulation, phase distortion, wave guide, granular, wavetable, etc) you'll be able to decide which method better suits the sound. There is no absolute "better" or "worse" here.