As a bassist who has played predominantly jazz for decades I can say understanding and being comfortable with the number system is necessary if you want to progress to be an advanced player. Here are just a few reasons why:
It allows you to associate chord progressions to any key and be able to transpose songs quickly and easily.
It is a standard and accepted method of communication between jazz musicians, i.e.: “Does this blues have a 2-5-1 or a 5-4-1 turnaround?”
It is part of the vocabulary for the understanding of jazz harmony and theory.
If I were to think about it more I’m sure I could come up with a few more reasons.
Now that’s not to say that it is all a numbers game. We must still be able to relate to notes and chords on a letter system as well and know our 12 keys and cycle of 5ths intimately. As a jazz musician if I’m playing say “Take the A Train” in C I am thinking of the letter chords:
C / / / | / / / / | D7 / / / | / / / / |
Dm7 / / / | G7 / / / | C / / / | Dm7 / G7 / ||
At the same time I am thinking:
I - II7(or V/V) - IIm7 - V7 - I - IIm7 - V7
If I have to play it in Ab it’s easy:
Ab-Bb7-Bbm7-Eb7 etc.
EDIT: It has been rightfully pointed out to me by @Lazy that my answer focuses solely on the harmonic aspects of the numerical system. My reason for this is because harmony is so much more often discussed as numbers than melody by jazz musicians. However I do think the numerical system for melody is just as important to learn and understand. There however is a bit of a vagueness to using the numerical system for melody in jazz. The reason I say that is conceptually some musicians may choose to use the numerical system based on the key of the song, i.e. in the key of G, A,B,C is 2,3,4. In the same key over an Am chord some may choose to think of those 3 notes as 1,2,3 of Am. The reason for that is jazz musicians are primarily improvisers. There are many different improvisational concepts but among the most used is to think of notes as the chord and passing tones of the chord being played and not the scale degrees of the key. I am among those who conceptualize melody based on the individual chords. Take “All The Things You Are”. It starts on the VIm chord with a 1 in the melody. To me it is a 3 in the melody of the VIm chord. It helps tie in the melody to the improvisation much better to me. That’s not to say it’s right or better then the other system, just another way of looking at things. This is something to be aware of and keep in mind.
Bottom line, knowing the number system will benefit you for the reasons listed above AND it will make you a better musician in the long run.