David Schiff MM (Manhattan School of Music, 1974), DMA (Juilliard, 1979) labels him mathematician:
I would propose a different model of Boulez's career: the typical trajectory of a mathematical genius. Like many mathematicians, Boulez had his great ideas in his early twenties, when he burst onto the Parisian musical scene as a radical composer and a scathing polemicist who intended to rewrite the history of twentieth-century music to justify his own innovations.
Wikipedia doesn't specify how much math he studied.
The following year he took classes in advanced mathematics at the University of Lyon with a view to gaining admission to the École Polytechnique in Paris. His father hoped this would lead to a career in engineering.
Ostensibly lacking any music degree, Damien Thompson (BA in History, Oxon. PhD in sociology of religion, LSE) wrote on 2011 Sep. 24:
I mustn’t mock, though, because after listening to his music in preparation for Exquisite Labyrinth, a Boulez festival at the Southbank Centre next weekend, I think I take him at his own estimation: i.e., he’s a genius. Alas, it’s hard to explain why in terms that he would find intellectually satisfactory. I have a CD of Boulez’s breakthrough piece, Le marteau sans maître (1955), played by his crack-shot Ensemble Intercontemporain, which comes with ‘helpful’ liner notes. Helpful if you have a degree in applied mathematics and/or linguistics, that is.