My friend got really sick with them on the day she was supposed to leave the hospital with her new baby. She ended up having to stay and have them surgically removed and she had to spend nearly a week in the hospital without the baby. She wasn't able to breastfeed during this period because of some of the medications she was on and the whole experience was pretty hard on her.
She had just had a c-section and claimed the pain from the kidney stones was worse. She had also been taking supplemental calcium in addition to her prenatal vitamin throughout her pregnancy which her doctor suggested may have made them worse. He recommended that if she wanted to continue to take calcium that she take calcium citrate (like Citrical), not calcium carbonate (like most other vitamins) - especially since she had a family history of kidney stones. Oh, and she'd been craving Coke her entire pregnancy and would have it as a 'treat' when she really wanted it. Of course, her doc said dark sodas also contribute to them.
Needless to say, I'd say get them taken care of. I can't imagine going through what my friend did and avoid that at any cost.