I was a hardcore fitness person to begin with.
So after #1 (vaginal delivery), I was back at it, full speed at 4 days post partum.
After #2 (C-Section), 2 weeks post partum..staples and all.
With both kids, I worked out up until I delivered, making very few adjustments to my intensity (i.e.: walking on the treadmill is not "working out" to me...).
Planks will be your best friend, so start them early!
Doctors know VERY LITTLE about fitness and pregnancy, except that fitter moms recover more quickly. So if you know your body, I would suggest you go with how you feel, rather than when your doctor gives you clearance.