Why do famous conductors rarely conduct ballet?
However, many famous conductors spend a large chunk of their time conducting operas beside just symphonic works.
TL;DR: Just skim the headings. The rest is sourced quotations supporting each point.
Few famous conductors have worked in dance with any frequency over the last half century. 
Other quotations below support this as well.
The conductor is often secondary to the choreographer and there may be multiple conductors for one ballet production.
"If one presumes that one of the goals of becoming a conductor is to deepen your interpretation of the repertory," said James DePreist, the director of conducting and orchestral studies at the Juilliard School, "conducting for ballet does not do that, because it's about suiting the dance and the demands of the choreographer." 
a ballet company's complicated schedule and long runs of repertory mean that multiple conductors may be involved, even in a single evening, lessening any individual's control over the orchestra. 
There is also less room for a conductor's ego; something in abundance especially in symphonic performances.
The ability to collaborate — both on a personal level and with another art form — and perhaps even subsume oneself, seems ultimately what a ballet conductor most needs. 
No one goes to the ballet for the conductor. 
These conductors’ names are seldom known to even the most ardent dance fans, and they’ll never attain the public persona of a Gustavo Dudamel or a Valery Gergiev. 
Recognition is rare. And the question arises: Given other options, what conductor would choose to conduct ballet? Very few. 
Working with dance (unlike opera) is not part of the typical university or conservatory curriculum, even at a school like Juilliard, which has a dance division. 
When you’re doing a conducting program, you’re exposed to symphony and opera conducting, but ballet is not something that is taught anywhere because no university has a professional dance company with a live orchestra that you can be trained on. 
you have to learn from someone who knows the medium since there’s no academic training available. 
Ballet conductors also need a working knowledge of ballet vocabulary and terminology. And the best have a real eye for movement. 
When a ballet conductor gets sick, it’s hard to get a conductor to fill in if they haven’t done that particular production because there’s a lot of information that’s not written in the score. 
As a ballet conductor, you need to hone a technique that allows you make changes, sometimes drastic ones, without confusing the orchestra. That requires a special set of skills because the orchestra has to be able to react immediately. 
Ballet, she says, “is such an enormous field to learn, I think many conductors are simply afraid of it. It takes years to understand dancers.” 
But what I do have are a very particular set of skills; skills I have acquired over a very long career. —Bryan Mills
"Generally in the profession it's safe to say that there has been and continues to be a general bias against conducting for dance," Mr. DePreist said. "You might have a reputation as a highly skilled musician, but the fact that you are not able to exercise your creativity counts negatively against you, and the industry has frowned upon conductors who have begun in ballet trying to make a transition to the concert stage." 
the president of Juilliard asked me up to his office and said to me, “I hear you’re conducting for ballet.” Being the only recently graduated Juilliard conductor who was working at the time, I thought he was going to be thrilled for me, but he said, “You’ve just thrown away a very promising conducting career. I’m really disappointed.” 
From what I can see, there’s uncertainty among music professionals about whether ballet conducting is a legitimate enterprise. 
many symphonic conductors believe that music played in a concert-hall setting must, by definition, be better and more true to the intent of the composer than music played for the ballet. 
Few companies “can afford to work consistently with a permanent orchestra and conductors.” 
Orchestral music and opera have much larger audiences than dance, attracting far more money from philanthropic sources, performances, recordings and videos or DVD's. As a result ballet companies, always short of cash, are unlikely to provide union contracts and musicians' salaries comparable to those offered by opera companies or orchestras. 
Well what an interesting question.
At first I was unsure that it was actually the case but a quick survey of who conducts for various ballet companies and on various tours suggests - to me anyway - that the OP has a point.
I think this will end up being too opinion based but my thinking is as follows.
For most orchestral concerts the conductor is "top dog". He gets the lions share of the applause (often rightly so as he has made the performance what it is) and most of the credit. When there is a soloist - for a concerto say - then that balance shifts, slightly, towards the soloist but ususally that is only for a part of a traditional concert and the rest of the programme all belongs to the conductor. The solost may also help the programme in general by attracting more audience members.
I consider the same to be true of opera. The director and the conductor are again lauded as top dogs. People will be attracted by big-name singers but the final plaudits will almost always be for the conductor (perhaps with the director) at the end.
But ballet appears to be different. The music is a supporting component of the spectacle which is all about the movement on-stage. Yes there is applause for the musicians and the conductor but thats more of the polite variety. The audience assesses the performance almost entirely on the on-stage action rather than the musical refinement. I'm not saying that this is fair, I'm just saying that is how it looks to me.
So I suppose what I am saying is that for concerts and opera its about the music but for ballet it is much more about the dance. Unfair perhaps, but how it is.
My mother was a ballet dancer and said that on one occasion the conductor was Thomas Beecham. She told me it was terrible - he only cared for the music and wasn't interested in the fact that dancers are constrained by things like gravity and mass.
And that's the problem with ballet - the conductor is effectively just there to keep the orchestra in time with the dancers, and to h*ll with the music. Some ballet companies use pre-recorded music, and I've no idea how they manage if things go slightly wrong on the stage.
Some conductors started their careers in the ballet - Pierre Monteux worked with Diagilev's Ballet Russes (and conducted the premiere of the Rite of Spring) is a prime example.
Since speculation appears in order for this question, I think there are a number of reasons, some more legit than others.
First, conductors do program suites, excepts, and sometimes full acts from ballets in orchestra concerts. These are from scores of composers who are considered to have written "the best" ballets, e.g., Ravel, Prokofiev, Tchaikovsky, etc. Keep in mind that full ballets are long. So when you say "conduct ballet," we should distinguish ballet music alone from fully staged ballets with dancers.
Ballet goers of course love the music, but focus on the dancing. There are lots of rehearsals and ballet conductors need to accommodate the dancers to a great extent. It's a skill unto itself, like conducting opera.
The top conductors are in great demand and I'm sure enjoy (and deserve) the big fees and being seen by the audience on the podium. Given a choice, I'm sure they would take conducting opera over ballet.
The repertory of good and great ballets is not huge. You can see this in the ballets that the major companies schedule -- lots of chestnuts, repeated. Some pretty good ballets do have some, shall we say, less interesting parts that don't stand on their own very well without the dance.
A top-notch conductor would want to lead a first-class ballets companies and first-class ballet orchestras. There are not that many of them. Also, ballets seasons tend to be short, for example, tacked onto the end of opera season at the same venue, so less oportunities.
You can program opera aria and even complete scenes for orchestra concerts with guest singers. A "guest ballet dancer" at an orchestral concert won't work -- for one thing, there's no room to dance! We have full concert performances of opera, but full ballets without dance are a rarity.