It's a little early to worry and the pg test is probably too early as well, but I wouldn't freak out just yet.
And if the ex is the kind of guy to say something like that, maybe you should go ahead and tell him just to get him out of the way! It takes two to tango. He could always use a condom, or God forbid, keep it in his pants.
Sex is a big responsibility. If you can't deal with the potential consequences, then don't do it. A single mom with two kids has enough on her plate; you and the ex should both think about what you're doing if neither of you are prepared...
Sorry, you probably didn't want a sermon, but there are so many people out there who can't even have kids, that I get a little perturbed when somebody says something that tacky about having one. He should grow up and he's probably not good enough for you or your kids anyway. Buy a vibrator! ;-)