I know you say the A above Middle C when refering to the A that is in the same Octave as Middle C in the Treble Staff, or you say the A below Middle C when referring to the A in the previous Octave on the Bass Staff, but what about if you're referring to the A in the next or previous Octave and so on? Do you say two As above Middle C or two As below Middle C or something like that?
This is an issue of what we call octave designation.
There is actually an international standard here: called International Pitch Notation (IPN), it labels Middle C as C4. An octave above Middle C is C5, an octave below Middle C is C3, etc.
In short, the C's octave range is in play until the next C changes the octave register. In other words, from C3 to B3 is all X3; it's not until C4 that pitches enter the X4 octave.
The G below Middle C, for instance, is G3, because it's within the C3 to B3 range. The G at the top of the treble clef will be G5.
Lastly, sometimes accidentals can be a little tricky. Imagine the B♯ that's enharmonic to Middle C. It's raised from B3, but it's enharmonic to C4. In this case, we focus on the note name. Since it's a chromatic alteration of B3, it's B♯3, even if it's enharmonic to C4.
I should also mention another system, that of Helmholtz notation, which uses a mixture of capital/lowercase letters and sub/superscript slashes:
It functions in a very similar way that the above IPN does, it's just a different labeling system.
In my experience, C4 as Middle C is very much the standard.
In addition to Richard's answer you might also consider MIDI note designations which work similarly. In MIDI there are 128 notes split into 10 octaves that start with C0 (#0) and going to G10 (#127) with every octave designation starting on C and moving up every 12 notes. This would put middle C at C5 (#60) rather than C4, and in IPN that makes octave 0 into octave -1. Of course, a normal 88 key piano can only play from note #21 (A1) to note #108 (C9), but if you're looking to compose using the computer can can be helpful to know these designations.
Sometimes there is benefit in losing some precision for the sake of simplicity. There are lots of good answers here but in rehearsal and when talking to students I find myself using a simpler way of communicating which octave I mean. I believe I learnt this from my first piano teacher.
The C in between the treble and bass clefs is middle C; the C in the middle of the treble clef is Treble C; the C in the middle of the bass clef is Bass C. There C on ledger lines above the treble clef is High C, the one below the bass clef is Low C. This gets 5 octaves which works well for voice and most instruments, especially in early levels.
To refer to non-C notes, I use above and below referring to the nearest C, so "the F above Low C" or "the G below Treble C."
I find this the easiest for spoken communication especially with students I work with a lot. For writing I generally default to IPN in Richard's answer.
If you'd like to name a note relative to middle C, it would probably be "2nd A above/below middle C". Many articles on English Wikipedia provide this kind of naming together with International Pitch Notation one.
Middle C is C4 in IPN so its octave can be called "fourth". However some MIDI-related software shift octave numbers up by one (probably because MIDI note 0 is C-1 and they want to display it as "C0" instead), for example middle C is "C5" in FL Studio. Some electronic keyboards have the opposite shift and middle C is labeled "C3" on them. This variation can lead to confusion.
Helmholtz notation provides different octave names. From middle C and up octaves are one-line (or once-accented), two-line (or twice-accented) and so on. Octaves below (from higher to lower) are called small, great, contra, sub-contra. No number variations, but naming of lower octaves isn't very intuitive.