First, an executive summary:
The primary point of these quoted passages is simply that any sequence of transpositions and inversions, no matter how long or convoluted, is ultimately the same as one transposition or inversion.
So, as discussed in the comments to the question, the meaning of X can be a single pitch class, but, more importantly, X can also stand for a set of pitch classes. We're rarely too interested in transpositions or inversions of a single pitch class, the primary interest is in sets. T5 of PC6 is PCe, because adding 5 to 6 gives 11. But, we can also more usefully talk about T5 of the set  as being [8e0]; we just add 5 to each PC in the set to get the new set.
The reason that transposition and inversion are being discussed so extensively is that those two operations are essentially the definition of set classes. A set class is the family of all PC sets that are related by transposition and/or inversion. Any two members of a set class—such as my examples  and [8e0] above, which are both members of the set class (014)—will be either transpositionally or inversionally related (in some cases, both). In my example, [8e0] is T5 of . I could just as easily say that  is T7 of [8e0].
A final note about pitch class set inversion: the symbol that Rahn (and many, many other theorists) use is TnI, and this is technically two operations, not one. First you perform an operation called I (remember, as Rahn notes, we always go from right to left with our operations), which simply flips a PC set around PC0, then you perform Tn. Nowadays, theorists tend to think of this as a single operation—for example, in the most recent edition of Straus' extremely important Introduction to Post-Tonal Theory he largely eschews the older TnI notation for In notation—and this is generally going to make your life easier when trying to understand these compound operations.
Rahn's primary point in these sections you quote a sequence of transpositions and inversions can always be reduced to a single operation. The easiest examples are when two different transpositions are performed in sequence. If I perform T5 on , and then perform T1 on the result, then I get this sequence:  -T5- [8e0] -T1- . The end result is precisely the same as if I had just performed T6 on the original set:  -T6- . This is because 5+1=6. Any sequence of transpositions will be the same as the sum of all the transposition numbers. Doing T5 followed by T2 followed by T4 is the same as T5+2+4, which is T11.
Where Rahn's discussion becomes particularly useful is once we start adding inversion into the mix, which, as he shows, introduces subtraction into the situation. This is where it's initially much easier to just imagine a single PC for X, and only move on to thinking in terms of sets after you get it. TnI of any PC is the same as the PC subtracted from n. For example, T2I of PC 5 is 2-5 = 9. Rahn says it slightly differently (but equivalently): TnI of X is equal to -X + n. -x is just the mod 12 complement of x. So T2I of PC 5 is -5+2. -5(mod 12) is the same as 7, and 7+2 is, again, 9. The same thing looks a bit more complicated when X is a set, but it ultimately works out entirely the same way: T2I of  is [78e], because -3+2=e, -6+2=8 and -7+2=7 (the order then gets switched to put the set in normal form, but that's just for ease of identification).
Rahn then notes that when combining multiple operations in a sequence, any pair of operations has to be one of four things:
- A transposition followed by another transposition. As noted in my answer already, if those transposition are Ta and Tb, the the result is the same as just doing Ta+b.
- An inversion followed by a transposition. Rahn does the math and reveals that for any inversion TaI followed by transposition Tb, the result will always be equivalent to Ta+bI. (Note that this is not the same as #1, in which the result was a transposition.
- A transposition followed by an inversion. The math says that any transposition Ta followed by an inversion TbI will be equivalent to Tb-aI. Again, the resulting operation is an inversion, just like in #2, but with a subtraction of the two indices rather than addition.
- Finally, the compound could be two inversions, call them TaI and TbI, and the result is the same as Tb-a. In other words, two inversions will always be equivalent to a transposition defined by the difference of their indices.
So, when we have an even longer string of operations such as T11I(T7(T0I(T2(T5(X)))), we can follow these rules to reduce it to a single, simple operation. Remember, we always go right to left, or, if you prefer, from the inside out. The first operation is T5 followed by T2. That's two transpositions, so it's equivalent to just doing T2+5, or T7. Now our innermost operation is T7, and it's followed by T0I. Our rule #3 tells us what we need: these two operations are equivalent to T0-7I, or T5I. That's our new innermost operation, and it's followed by T7. So now we need to look to rule #2, which says this is equivalent to T5+7I, which is T0I. That's our second to last innermost operation, and it's followed by T11I. Now we have two inversions in a row, so we're talking about rule #4, which tells us the result is T11-0 or T11. All of that complicated string of operations is ultimately the same as just doing a single T11. This is easiest to verify by just plugging in a single pitch class, for example using PC4:
T11I(T7(T0I(T2(T5(PC4)))). T5 of 4 is 9, T2 of 9 is e, T0I of e is 1, T7 of 1 is 8, and T11I of 8 is 3. Or we could have just performed T11 on PC4, and sure enough, the result is 3. The fact that T11I(T7(T0I(T2(T5()))) is simply  takes a lot longer to verify, but it ultimately works out as well.
Let me know if I can further clarify anything.