Delay is an effect in itself, and also the basis of some of the other effects.
Delay is simply the effect of playing the input signal some time later than it was fed into the device. Delays can be as short as a couple of milliseconds, as long as a bar, or more.
Note that no signal processing is instantaneous, so every effect adds some small delay.
Usually, but not always, the delayed signal is combined with the original signal, because players want to hear what they play the moment they play it. Often it is mixed such that the original signal is louder than the delayed signal.
Originally delay was achieved using a loop of magnetic tape - first on improvised arrangements with a reel-to-reel recorder, and later on dedicated machines. The tape would pass through a record head, then a playback head, then an erase head. The timing of the delay could be adjusted by moving the heads, or changing the speed of the tape. Tape adds its own colour to sound, so the echo would have that added warmth.
Later, analogue circuits were developed to achieve delay, first with tubes, then with transistors, each of these adding a certain colour to the delayed signal.
Digital delay inevitably followed, and of course now we have digital delays which attempt to model the effects of the analogue devices of history.
When the delay is very short, the signals tend to interfere with each other in interesting ways -- see Phaser and Flanger.
When the delay is slightly longer -- close enough to still be considered "on the beat" -- you get what is known as doubling echo. This sounds similar to having two instruments playing in unison. The delay is simulating the slight difference in timing two human players would have.
Chorus is a variation on doubling echo, in which the delay varies -- the tape loop speeding up and slowing down continually. This means your "virtual" duet partner is slightly off time by a varying degree, and also slightly off-pitch by a varying degree. See the separate answer on chorus.
When the delay is longer still -- long enough that you begin to be able to hear it as a distinct note -- this is known as slapback echo. It's an effect frequently heard in rockabilly guitar, and the "skanking" guitar parts of reggae.
When the delay is even longer, delay becomes a melodic tool -- you are effectively duetting with your earlier self. Many U2 songs, e.g. "Where the Streets Have No Name", are built around this effect.
It's common to feed the delayed signal back into itself, so that you get echoes of echoes, each (usually) quieter than the last.
Using this fed-back delay on short delays creates a crude reverb effect. With longer delays it becomes a rhythmic/melodic effect. For example it's common to add delay to parts of drum patterns to create a complex rhythm part.
With analogue delay (and simulations thereof), each subsequent echo is not only quieter, but also more distorted. Dub reggae tracks often make prominent use of this effect -- look out for the effect where the engineer momentarily turns a knob so that the echoes get louder instead of quieter, surging and distorting, before he turns it back down again so they can die away.