Dear C.,
I think it's good for you to come up with a game plan now, because as you said, you are wondering how it's going to be when shool is out for summer.
You're just going to have to keep saying no. And if there's a day when you don't mind them visiting to jump, be very specific before they ever even get to the trampoline. Make them stand still and look at you while you tell them the RULES:
1. Tramp is for jumping ONLY
2. NO MEANS NO
3. NO fighting. NO wrestling. NO rough-housing. NO hitting.
4. NO zipping and unzipping. NO coming in and out.
5. When you say it's time for them to go home, it's time for them to go home. Period. (It doesn't matter what you're doing, where you're going, dinner or no dinner, whether you want family time or not, GO means GO).
5. If even ONE of the rules is broken, even ONE time, they will be told to leave immediately.
Make them repeat the rules to you before they ever get near the trampoline so there is no misunderstanding or them trying to argue with you that they didn't know or understand. They are both old enough to understand exactly what you are saying. There is also nothing wrong with you saying it's okay for them to visit for a bit before you start dinner (or whatever), but no trampoline. If they ask why...refer to Rule #2, above.
You might want to talk to their mother and make sure you have each other's phone numbers. She may have thought that you were inviting the boys all the time and them coming over so much was perfectly okay with you. Like the day you already told them no and they came back two minutes later saying their mom told them it was okay...That's when you could have picked up the phone and said, "I'm sorry. There must have been a misunderstanding. They boys just came back over saying you gave them permission to jump, but I had already told them not today."
You can be very friendly and nice about it, start off by saying you like the boys and they are very sweet, but somedays they get way too rough, and you know they are having fun, but sometimes they don't really listen to you when you say it's time for them to go so you can start dinner, etc. And I would tell her that one day you were down sick and they wanted to jump anyway. Tell her they are a lot of fun to have around sometimes but you have a 3 year old and your schedule runs differently having a little one and you'd just like to have a way to invite them to come and have fun and confim that it's okay with her first. Or, that way, she can call you and ask if they can come over. If you are sick or your kids are napping or you're leaving for an appointment, she'll know it's not a good time.
Trampolines are kid magnets for sure. I have a good friend who got one and a couple extra neighbor siblings were showing up and said their mom gave them permission. Turned out their mom didn't know they were over there at all. They were supposed to be somewhere else after school and you-know-what hit the fan. So, I would definitely communicate with the mom and have your steadfast rules. Let her know what the rules are too so that if the kids are given one hour to jump and they're back home in 15 minutes, it's because rules were broken.
Your rules are in place for safety reasons above all else and surely she should understand that.
I wish you the best!