It sounds like others posting aren't seeing that you say the lunch is "required" and you've already paid for it, for the kids attending. You can't just say "cake and ice cream only" if you have paid up for a lunch. (If you have not paid yet, can you back out and have this party at home at a time YOU want and not offer a full meal?)
If you are committed to this place -- I would tell parents very clearly and immediately that this is a drop-off party and if they want to stay, they are welcome to stay but that only the guest child's meal is paid for; parents must pay for their own meal if they plan to stay and eat. You will get some parents who just drop their kids, others who stay and do not eat themselves, and a few who will stay and pay to eat. Be clear, or some parents may not realize that this buffet is not open to all for free and may eat without knowing there is a charge. That would put you iin a very bad position of either saying nothing and paying for them at full adult price, or having to say, "Um, sorry, you have to pay for that food you just ate...."
Be clear too that the party is for invited kids and not siblings!!
I would go for 1:00 since you have paid for food; if you've paid, then ensure they eat by having it at lunch time. Be SURE the parents know there is a full meal included so they don't give kids lunch before the kids come!
I would not use this venue again. Sounds like they book a party then figure on extra bucks from parents who stay to eat. I would go for a party next year at home or in a rented room (not a restaurant/event place) from maybe 2-4 or 3-5 and just offer cake, ice cream and carrots/pretzels/etc.