I have only ever owned one audio interface and it was a Tascam something or other - which I recently sold. This device plugged directly into my computer with a USB. The device had 2 input channels .. and I could either plug in a microphone (3 pin) style cable or a regular 1/4 inch cable.
One of the sides also had Phantom Power which allowed a voltage to head towards my condenser microphone which allowed it to be powered and work! On the audio interface I could control the input volume of each input which meant that I could do some basic 'mixing' whilst recording which Guerrilla Home Recording told me was better because it would free up some 'usage' in the computer, and allow a smoother recording experience.
I could also plug my headphones directly into the audio interface and listen to what I was recording. Doing this into the pc would be crazy (I believe), as there would be latency issues i.e. by the time the computer had processed everything and outputted it to my headphones then there would be a delay.
The audio interface simply plugged into mac / pc and I could use it with any recording software I tried. It was not expensive and allowed me to make good quality recordings. Sometimes I had a problem with the input channel that meant it difficult to get the input right. A light on the device flashes red to tell me I'm being too loud.
Essentially all these things - aside from being a physical and tangible device that you can quickly change things on with dials, which is great - are why I used an audio interface. It allowed me to convert 2 input channels e.g. a cable coming from a guitar and a cable both powering and coming from a condenser microphone, at the same time, into a digital format that the computer could interpret and process - so that a digital recording was possible.
In terms of midi controllers I don't suppose having an audio interface would be very useful if you recording directly. As for monitors coming from your computer they are good at playing out what you have recorded - but I have always found a bit of latency on playing it like a normal amplifier i.e. pressing notes on a keyboard and hearing them play back. Though I think setting this up in another way is quite tricky. If you're recording channels at the same time - or even not at the same time - going through USB might lose some of the experience. The USB mic, etc. like you said is in a way also an audio interface so your question pertains more to the use of terms perhaps! I'm not an expert but I have spent many hundreds of hours recording with audio interface and midi keyboard (which I plugged in directly to the computer and never recorded simultaneously). So people use Audio interfaces when they want various channels coming in that they can easily alter for inputs, and also listen to directly for a smoother experience.
Hope something here helps.