Terminology
Every sound is composed of one or more sine waves.
From that group of sine waves, the one with the lowest frequency is called the fundamental, every other sine wave above that one is called an overtone.
Overtones that are integer multiples of the fundamental are called harmonics. The fundamental is considered a harmonic, the first harmonic, but not an overtone.
The characteristics of these overtones (like quantity, amplitude, and frequency) is what gives each sound a specific timbre. That's why a guitar and a piano sound different, even when playing the same note.
What is a "dirty" sound?
It's all about the overtones.
How many overtones. A sound with more overtones will tend to sound dirtier than one with less overtones.
Frequency of the overtones and their relation with the fundamental. From two sounds with equal quantity of overtones, the one with less harmonics (less overtones that are integer multiples of the fundamental) will tend to be perceived as dirtier.
Loudness of the overtones. The loudest overtones will have a bigger influence in the timber of a sound than the quieter ones.
So, it is about both quantity and quality.
These 3 are what I consider the most important dynamics that define the dirtiness of a sound.
There's a more in-depth, scientific, concept called auditory roughness introduced by Helmholtz in 1885 which takes other things into consideration, like amplitude fluctuations of the spectrum and pitch instability. You can read more about it here, here, and here.
Seeing and hearing overtones, dirty and clean sounds
(check your sound levels before playing these)
This is how a sine wave sounds like:
https://soundcloud.com/jcpedroza/sine-wave-in-c-523-25hz
This is how a square wave sounds like:
Both are at the same frequency (note): C 523.25Hz, but the square wave has much more overtones than the sine wave (the sine has no overtones and only one harmonic). Can you notice how the square wave is much more "dirtier" than the sine wave, which is much more "cleaner"?
Using software, we can "see" these overtones, and the waveform of the sound.
This is the waveform of a sine wave:

And here is the overtone content of a sine wave.

Here we can clearly see that the sine wave has only one harmonic, the fundamental, and has no overtones. It's that peak at around 523Hz.
Now let's see the waveform of a square wave:

And the overtones of the square wave:

The square wave has many more overtones! It sounds "dirtier".
We know that the frequency of the overtones and their relation to the fundamental can also carve how dirty a sound is perceived, it's not all about the quantity.
Here we have a sound with a fundamental frequency of 500 Hz and its first 4 harmonics at 1000, 1500, 2000, and 2500 Hz, and each overtone with an amplitude of 1/n. These are integer multiples of the fundamental.
And here we have a similar sound with a fundamental of 500 Hz, with its first 4 overtones, and each overtone with an amplitude of 1/n. The only difference is that the frequency of the overtones was slightly modified. The new overtone frequencies are: 995, 1515, 2020, and 2450 Hz, they are no longer integer multiples of the fundamental.
The first sounds cleaner than the second because in the first all the overtones are harmonics, all overtones are integer multiples of the fundamental. In contrast, none of the overtones of the second sound are integer multiples of the fundamental, which translates to a dirtier sound.
Making some dirt
We can hear and see how a "clean" sound becomes "dirtier" by adding each overtone individually over time.
This is a sine wave (clean sound) turning into a square wave (dirty sound) over the period of 20 seconds, same note C 523.25Hz (WARNING, the loudness at the end is significantly higher than at the start, be careful with your local sound levels!):
Notice how the sound becomes dirtier as we add more overtones, as the sound turns from a sine wave into a square wave?
Here we can see how overtones are being added over time. From top to bottom is: at 0, 5, 10, 15, and 20 seconds:

Dirty and acoustic
This difference in overtones can be found in acoustic instruments too. Let's look at the overtones of 3 instruments: a clarinet, a piano, and a trumpet, all played in C 261.62Hz.
Clarinet:

Piano:

Trumpet:

We can see that the trumpet is much more rich in overtone content than the clarinet and the piano; the trumpet is "dirtier".