I teach young children piano and have noticed that my students who have started from age 6 on up have more long term success than the ones under 5. For example, it takes a child who is 7 or 8 way less time to learn basic musical concepts (like rhythm and notes) than a 4 or 5 year old. A 7 or 8 year old could possibly advance through two book levels in a year with lots of practice.
My parents bought a piano when we were very little and had lots and lots of wonderful music on records. This was great exposure and having the piano really got me curious about learning. Infact I didn't start official lessons until I was 14!
I would recommend group music lessons at this age like Kindermusik.....so there is good exposure without the pressure of private lessons.
Also when they are older you can see more clearly if your child has good chemistry with the teacher.
Mostly for me teaching really young students has felt more like a weekly music work shop than actual lessons.
I hope I'm not sounding too harsh but this has been my general experience.
That being said, I do have one 8 year old boy who started lessons at 4.5. He is doing well and his mom committed to working with him daily for the first 2 years until he could practice on his own. There were some tears shed in the beginning as her 5 year old was not always in the mood to practice....but I have to admit...he's a darn good piano player now. Would I have done than with my daughter?
I don't agree with any tears being shed around music at all....so no.....I couldn't take that risk. Also this particular mom is not having the same success with her younger son.
So I guess....it depends on what you want.....My goal is to cultivate a life long love of music in my students.....and to me starting too young can sometimes be a little complicated.