A synth is a machine that generates (synthesizes) sounds. There are many different types: old-school 80's style FM synthesizers, even more old-school modular synths like Moogs, software synthesizers that run on a computer, etc. Although some have a keyboard, that is not part of a synthesizer per se, that's just the controller.
A controller is a thing that can send note and other information to a synthesizer. The vast majority are keyboard controllers, but there are also drum pads, software, breath controllers, etc. All a controller does is send performance information.
Many controllers, especially keyboard ones, have built in synths, but they can usually also be used via midi cables to control other synths. For that matter, the synthesizer inside such hybrids can also be controlled by other controllers if desired.
So, if you get a keyboard controller, you will need some kind of synth to control with it if you want sounds. Although there are many awesome hardware synths out there, they are expensive, and there are fantastic software options which are relatively cheap if you already have a computer. Buying a hybrid might give you somewhat more portability, but if you have a laptop, it's not a huge difference.
EDIT TO ADD: for the most part, musicians end up collecting many different synths and controllers, especially synths. Different synths have vastly different sounds and strengths. Some have exotic sounds, some specialize in recreating traditional instruments, some favor pads while others favor leads. The majority have at least a few of all of these categories and more.
