This is my big gripe! I had one party for my son where if I went over it was going to cost me $200. On the invitations, I simply stated something to the effect of due to space limitations, no siblings please. I got one nasty response from a single mom, who never told anyone her situation. She was not a close friend and the year before she brought her older one too without telling me and then the older one kept interferring with the pictures I was trying to take of my son. So, no I didn't feel bad and I held it on a weekend, gave plenty of notice so she could have easily scheduled a play date or asked a relative to take her older daughter.
It is your dime and your party - you need to do what is best for you. If you make an exception, make sure it is for the right reason, close friend, etc... just not because you feel bad.
When they say can I bring little Tommie too, just say something to the effect of "Gee Barb, I wish I could but we are really limited in the number we can invite (if you want add something like I invited the whole class/my child is such the social butterfly/etc). You know since Sue has little Bobbie and her daughter Janie is invited too, why don't you see if you can do a playdate/carpool combination so each child gets to spend time with their friends since this is a dropoff party. Thanks for understanding!"
Be prepared for some to just show up with sibling(s) in tow and be prepared to stop them or have DH or a close friend play the heavy. "Hi Betty! Please feel free to leave Erin here and you & Bobby might want to run to the park/mall down the street while Susie's party is going on. You are, of course, welcome to stay, but little Bobbie will have to wait here with you while the party is going on." Of course, allow the appropriate pauses for responses (oh, we are going to run over to the grocery store and stock up for the week), so you may not even have to get to the last sentence.