You should listen to your doctor on certain things like if your blood pressure is high go to a low salt diet, try and do everything healthy for yourself and your baby that you can. I did get preggo at 39 with my one and only and he was premature, they knew it could happen due to age and blood presure issues so i was watched and helped along.
Do not dismiss things doctors say, just take it with a grain of salt and look into it yourself. My son was a 90% chance of having downs, my doctor did not want to do an annino due to that risk, so we went for a high end ultra sound and he then was listed as a 10% chance of downs and he did not have it.
Since you have already had a baby with no health risks to you or the baby your chances are good. But take precautions eat well, maybe take up yoga to get your body in better shape so you are stronger and eat the best you can.
Good Luck, your daughter with have someone to fight with besides you now! lol