I highly recommend House of Bounce, Oxbow Park, and the soldier's field park. There is also a nice park and playground at Silver Lake. Walking around silver lake is also fun - bring bread to feed the geese! (I hate the geese. they are messy and gross...but tourists love them). the House of Bounce has recently expanded and relocated - my kids LOVE it. (4 yrs and 20 months) There is a toddler section and a bigger section...a two year old would be able to use both but you may want to go at a time devoted to toddlers, just so the bigger kids don't bounce on him/her.
Thursdays on first and third is a lot of fun too...there is usually entertainment at night. Someone mentinoed Quarry Hill Nature Center - a great place to visit! There is also a park/campground called Chesterwoods with nice hiking trails, fishing piers, and a nice beach. I think it is about $5 a car to get in there.