When my older son was that age we had 3 morning commitments: a class at the YMCA, a class at the park district and a play group. I thought it was very healthy for him to be doing those things. When he was 3 he began preschool. It was a transition, as I was a SAHM at the time, but it was so good for him. The first couple of weeks were hard, but he loved it.
He is now in kindergarten. His younger brother is almost 3. I started working full-time last fall when little brother was 2 1/2. He is in daycare full time and doing great!!!
You have to do what works for you and your child. I believe it is your responsibilty to help him grow both academically (age appropriate, of course), emotionally, socially, etc. MyGym, storytime, playground, etc., sounds great! She is just a little over two, so that is just so age appropriate. I really would consider preschool next fall. It doesn't have to be everyday. My son went 2 mornings a week. It's just such a great way for them to grow socially without Mommy there. They learn to play more cooperatively, negotiate, listen to an adult who is not Mommy. It's just such an amazing experience.