I'll describe the approach I took, and you can see if any of these activities appeal to you.
Kindermusik worked really well for one of my children when he was a toddler. I still sometimes sing, "Toast, toast, I like it a lot"!
For both of my children I kept a box of safe instruments for them and their friends to play. The sunshine drum, a tambourine, some jingle bells shaped in a circle, a wood block, a one-octave xylophone. I took the batteries out of their toys because the tinny electronic sound got on my nerves. I did not include anything in the instrument play box that had an annoying sound, or they could damage, or that could harm them. When playmates came over, they would each choose an instrument and then make a parade with "music."
We had some good children's songbooks, and we cycled them through the bedtime reading, and sometimes sang from them during the day as well.
I took them to children's concerts, orchestra concerts, puppet shows for their age range (which often included music), ballet and other dance performances.
I made sure that all their musical experiences were positive -- so I didn't take them to highbrow concerts where they might have gotten on people's nerves, or they might have been uncomfortable. At the ballet, we sat next to an outside aisle to be able to make a quick exit if necessary. I made sure they had had plenty of exercise before we went, and I had them run up and down the aisle, or go up and down the stairs, during the intermission.
I would get permission to take them to dress rehearsals of dance performances that didn't include a matinee performance, because that way they didn't have to stay up until 10 or 11.
For my first child, the first instrument was the recorder. For my second child, the first instrument was the viola (it was actually a very small violin, strung like a viola). The recorder teacher was from Austria, and had many years of experience teaching recorder to children. The viola teacher was a Suzuki teacher, and teacher trainer.
My younger son LOVED to dance. Even at the age of two, he would work out his choreography, and then sit us all down to watch. His favorite music to dance to was Tito Puente. He was given a toy DJ sort of thing, where he could get different drum-like sounds by hitting different areas. That was pretty much the only toy he could play with by himself (without interaction with another person). He liked to play his brother's drumset from age two. He couldn't do much harm, because it was a hand-me-down from a hand-me-down.
I never allowed my children to experiment with my cello. I just wasn't comfortable with that.