What is tritone substitution? How can this device be applied to improvisation and composition?
Tritone substitution is as it says. The substitution of one chord for another, that is a tritone away from the one being substituted. Thus a V7-I ( G7 - C ) becomes Db7 - C. Because the Db is a tritone, or 3 tones away from the G. Exactly half way, as it happens. G7 is spelled G,B,D and F. Db7 is Db,F,Ab and Cb. The two common notes of F and B (Cb), being a tritone apart in themselves, swap places, the F (b7) of the G chord becomes the maj. 3 of the Db, while the b7 of Db (B/ Cb) becomes the maj. 3 of G.
Thus the chords swap over. Resolution is often from one note in a chord to a note one semitone away in the resolving chord. Going from the Db to C notes, and the Ab to G notes bring a resolution (as actually,the F and the Cb do as well)- all notes being one semitone from the target.
As far as composition and improv are concerned, tritone sub. can be used in a perfect cadence situation, swapping a V7 - I for a bII7 - I. It introduces non-diatonic notes into the harmony, whilst one can use diatonic notes (or the appropriate non-diatonic) in the melody. It also works with an interrupted cadence. Using ,say, C -E7 -Am as an example, t.s. the E7 for a Bb7 to get to Am.
In a jazz situation, often only two notes are played for a 'chord', so in the first example, B and F could be played to get to Cmaj. That alone isn't a t.s., but the bass for instance, could play a Db. Then it is.
Tritone substitution is the substitution of one chord (almost always a dominant 7th of some sort) for one with a root a tritone away. For instance, you could substitute Db7 for G7 because Db is a tritone away from G. This works because the important notes in the chord that determine what it leads to (the third and seventh) are the same in both chords - in this case, they would both be B and F. Tritone substitution is used to make changes sound more interesting.
As the others have already given good explanations, here's a beautiful fingerpicking tune which gives a good impression of how this sounds in practice: (not really "jazz" though)
This is a true tritone in chord changes. Very rarely seen. I wish it would work to auto-skip the tune to about 2:50 where the chord change (G#-D) happens. With YT links it works, but with SE's embedding code, it does not work at all.