The consonance of the 4th interval depends on the context, as several music theory sources describe. Why isn't that idea generalized to all intervals? Let me illustrate this comparing what I have in mind for the 4th and the 5th.
If the interval G-C is played, the G can be heard as the tonic, which makes the ear expect a pull C-B to resolve that suspended interval (it's like there's a hidden Gsus4). If the C is heard as the tonic, everything seems stable (most references categorize the 4th as a dissonance even if the C is heard as tonic, when the 4th interval is made with the bass note, which means a second inversion triad. Maybe because I wasn't exposed enough to common practice music, my ear don't feel the extreme urge to resolve the so called instability of a second inversion triad)
If the interval F-C is played, when F is heard as the tonic, we have a stable interval. But wouldn't the reasoning be exactly the same if C is heard as the tonic? In this case there's an expected plagal pull F-E and shouldn't the books list the consonance of perfect 5th as an "interval that depends on the context"? Why people put just the 4th in this special place?
Vincent Persichetti (Twentieth Century Harmony) also categorizes the augmented 4th as an interval of this type and goes even further stating that the consonance depends on the surroundings. Some passages are exhibited here:
Don't all intervals have their degree of consonance dependent on the surroudings and context? A major 2nd sounds more consonant if all that came before was a sequence of minor 2nds, for example.