A synthesizer is simply any instrument that uses some purely electronic means to generate musical tones. Most synthesizers use a keyboard to allow the performer to play the instrument as they would a piano or organ, however there are interfaces available that can sense the frequency produced by more traditional instruments such as guitars, and synthesize notes or sounds at the same frequency.
The first "synthesizers" were simple sine-wave generators first invented around the turn of the century, used for transmitting tones across telegraph wires. With the invention of the vacuum-tube amplifier in the early 1900s, and the development of additive synthesis (the synthesis of two or more individual tones simultaneously which are then summed to form a more complex timbre) shortly thereafter, the synthesizer became a practical if someone sterile-sounding instrument, with incarnations such as the theremin and Martenot coming to market in the late '20s.
Synthesizers started becoming popular in mainstream music with the development of the Hammond Organ, a self-contained console synthesizer designed to substitute for the expensive, large pipe organ. These consoles became very popular with poorer African-American Southern Baptist and Southern Methodist churches, and were subsequently introduced into R&B by artists like Ray Charles who grew up in the gospel tradition, as well as into country-western genres also popular in the Deep South. The sound of the Hammond has made it a musical icon, and the vintage units are in high demand by studios and touring acts, as well as in sports venues like baseball stadiums where it's traditionally played from the press box for charge yells and other "cheerleading" purposes.
In the 70s and 80s, the development of the solid-state transistor and the integrated circuit made synthesizers much smaller and more flexible, able to be easily customized by their settings to produce a wide variety of tones from trumpet-like blasts to smooth sine waves. This purely synthetic tone, not aiming to reproduce anything in particular, became popular during this time, with songs like the theme to Beverly Hills Cop making heavy use of the technology.
At the same time, sample-based synthesis also hit the mainstream, with keyboards having a range of more realistic tones produced by modifying the frequency of a set of sample sounds from real instruments. This system, and the library of tones available, became standardized along with the ability to digitally save and reproduce performances, with the development of the Musical Instrument Digital Interface or MIDI, which takes the input signals from the synthesizer or other instrument and digests them into a text-based series of "attacks" and "releases" of various pitches, played by different instruments at different "velocities". The resulting data can be saved to a computer file, the data edited digitally using a sequencer program, then transferred to and played back by any device with a MIDI-compatible synthesizer.
Nowadays, virtually all electronic instruments and all major audio workstation software is fully integrated with MIDI, with additional support for "SoundFonts", files containing differing samples for the various standard MIDI instruments, allowing the same keyboard or computer synthesizer to sound dramatically different just by loading a different SoundFont. This allows for even greater portability; a musician can take both their sequenced MIDI track and the SoundFont they used to compose it from their home studio to their professional setup or to a recording studio, and know that when it's played back by the different equipment, it will sound exactly the same as it did at home with no surprises.
The SoundFont/MIDI technology also allows for musicians to capture samples of unconventional sounds not normally found in the "General MIDI Library", then "play" those sounds through the keyboard. This is a common technique for sound designers in movies, recording sounds like trees falling or animal roars, editing and combining them, then playing the sounds in sync with the on-screen events to create the sound effects.