When I'm practicing with a metronome, how hard should I be on myself before saying "OK, I got it down at this speed, let's increase the tempo."?
I'm going with "after playing 'a while' and getting (subjectively) comfortable with the exercise, make three more repetitions with absolutely no mistakes". Sometimes this turns out to be pretty hard, because lack of concentration kicks in.
So, how to deal with practicing with a metronome, regarding this particular aspect of "levelling up"?
I don't think practising with a metronome should be any different than normal, so my advice is that you should be able to play it flawlessly consistently before increasing your tempo. If you're having trouble with focus you could try playing smaller parts repeatedly, and then stitching them together. The short parts should allow for you to get into a rhythm and not need as much focus, without making the entire song as boring.
If you don't like the "N times with no mistakes" approach, then another approach you might try is the "slow, fast, slow, fast" approach. In that approach, "slow" is a speed you're comfortable playing with no mistakes, and "fast" is one notch outside of your comfort zone. (So the amount you change your metronome should be very small, if you think about it.)
If I may, the lack of concentration you point out is kinda the point. When you're able to play the piece without the majority of your mental concentration directly on each individual physical motion you'll be able to focus on introducing more nuance into the music. This is the goal of practice – to know the instrument so well that your attention is on making music, not playing notes. I'm not trying to suggest that you deliberately divide your attention, but just recognize that your wandering concentration and even a small sense of boredom is a necessary transition point in learning technically difficult music.
I find it very difficult to speed up by raising the metronome setting and playing along. I find it just adds to my confusion and panic. So I use the metronome just to stabilize the tempo in my playing (find the spots where I'm rushing or dragging, and even it out).
To actually increase the tempo is more "mental". I have to find larger rhythmic phrases to focus on, and ignore smaller details (letting the fingers take over). Sometimes I have to step away from the instrument entirely and just focus on reading faster, raising the tempo in my audile imagination.
It depends a little on how you're using your metronome to practice. My instructor once guided me to turn the metronome down to a speed where I was confident that I could play the piece 100% with no mistakes. When first learning a tune, this could be painfully slow. Each subsequent time I would sit down to practice this piece, I would start again at this tempo. After a couple of run throughs at this speed, I would increase the metronome by 10 BPM, after a couple more run throughs, another 10 BPM increase, until I was one or two "clicks" (increments of 10 BPMs) above the desired speed of the song.
After I played it a few times at a speed faster than the destination tempo, I would turn the metronome back down to the desired speed, and suddenly my technical accuracy would improve, since I started from 100% accuracy, and built up slowly to a tempo above the destination tempo.
I agree with another poster here that getting bored with the tune is actually a good thing. You want to be as comfortable as possible while playing the piece, not frantic in a "hanging on for dear life" mentality while playing the piece. It should seem effortless, not really difficult during the performance. Stepping up the BPM slowly should help in this effort.
I would recommend not using a metronome to "push" your tempo faster. I recommend using your metronome to set a goal tempo and measuring your progress towards that tempo. Then play the exercise without a speed constraint. It helps you focus more on learning the piece, as opposed to learning the piece quickly.
Once you have really learned the piece, the speed at which you play will be limited by your technical ability, rather than your knowledge of that specific music. While you can increase your technical ability by repeating the same exercise over and over, this is largely because you increase your technical ability by playing music at all. I prefer to practice without the metronome every day, while also making sure to play other pieces, especially music that I already know well. Just like your reading ability was developed more by reading lots of books as opposed to the same one over and over, your playing ability will be developed more by just playing a variety of music.
Since you won't be able to play significantly faster immediately, I would recommend placing your time checks less frequently than every time you run-through the piece, like every week or every other day. Just give yourself time to improve instead of constantly measuring yourself.
I'm currently working with an hypothesis: the number of mistakes must be less than the number of well succeeded executions. I have no scientific evidence for that, but the reasoning would be to avoid teaching something wrong to your brain or muscular memory.
This first paragraph doesn't answer your question, but thinking like that, you can imagine that there's a "threshold" where you're improving, and not going back. For example, I usually use 60 to 80%, depending on how hard the piece is, or how much time I want to spend for that. I mean, for the last N executions (usually N is 10 for me), I need to execute properly at least 60% percent of it. If I'm less than 60% right, I decrease the tempo by 5% (it also can change as you wish), if 80% or more of my attempts are correct, I can increase the tempo by 5%.
Notice that we have lots of parameters there, they all can be changed accordingly to your purposes. And unfortunately, you still rely on your ears to choose between a good execution and a bad one. Also this method is pretty simple, and it doesn't include the benefits that some "pushes" beyond your comfort zone may give you.
In the end it's exactly what you're already doing, but now you're measuring and can understand if you're improving or not. Use that feedback to refresh your parameters and manage your own "levelling up" criteria.
Being able to play a piece with no mistakes at a particular speed along with a metronome lets you know you can speed it up a bit and see how you do.
I find cranking it up by 5-10% each step works quite well.
