As I see it, he got a natural consequence of the rough play – he had to stop playing. Possibly because he was so engaged in having fun while being near Mom and Baby, possibly because Baby "won" over his getting to play where he wanted, possibly because he was just excited and a little out of control (4yo's do that), your request, though reasonable, hit him wrong. So then he needed consequences for acting out.
Once that moment of anger/outrage strikes a child (and generally an adult, too), listening stops. It's hard for anything else to get through. I'm not "excusing" this, just observing that it's true. In your son's case, his anger led to some inappropriate behavior. I personally find it admirable that you waited and gave yourself a chance to think, AND that your son had calmed down enough to be playing and listening when you went in to talk to him.
You both did some good work there, from your description. (And in my observation, the child eventually comes up with a sincere apology a little later, maybe an hour or a day, when calm is reestablished and a heartful feeling of connection/forgiveness follows.)
You had a couple of options at that point besides just looking for a consequence:
You could have asked your son to talk about his behavior, consider what other options he had, and ask HIM to choose a consequence. Many kids will do an amazing job of this if given a little time and support. My grandson was able to do this by the time he was 3 or so. And many kids will find a consequence that's not only appropriate and creative, but harder on themselves than what a parent might have chosen.
But another option you might consider "next time" would be not to send the rowdy play to another room, but to pause the play and ask him to notice that the baby almost got hurt. You could ask your son right then what HE would do to fix the problem. He could have come up with any number of options that would have met his emotional needs to feel included in, and because the solution would have been his, he would have been invested in making it work.
These techniques are taught in the best parenting book I have ever used, How to Talk So Kids Will Listen, and Listen So Kids Will Talk, by Faber and Mazlish. The book is abundant with real-life examples of how parents helped set the conditions for the child to address the problem himself. And though we don't usually think about young children in these terms. they can be creative problem solvers. Plus, kids are more invested in solutions they think of themselves, and are more likely to work at making them a success.
This could be a really useful option for those times when consequences seem to happen too late or are hard to figure out.