N.B.
He's just getting to the age to play with others. Family is all he's had up until recently, and socializing with outsiders without Mom standing right there is really hard for some kids. I wouldn't worry, or push. He sounds like a normal, happy, well-adjusted kid. Maybe because he has a speech problem, you're too finely tuned in to every behavior. As he gets older, group activities will help draw him out and let him feel less "different." I think you're doing great with him.