Like some others said, I would assume they meant it was OK to bring a baby. You stated its a child's birthday party to begin with...so maybe they have a baby of their own, so they have baby toys or some sort of set up that works for their baby (a swing with buckle straps, lots of playballs, bubbles, little push toys, etc). I have such a yard (and an area indoors) as I do home daycare and all of my crew are usually age 3 and under. My house is a great place for family parties as we have little ones on up to 10 year olds in the family!
Sounds like a very family friendly gathering!