C.,
My parents had 3 biological children, and adopted one. I am the oldest. I love my siblings with a fierce devotion that my husband thinks borders on the absurd. I also have 30 cousins, many of whom I am still close to, and we all grew up in the same town, went to the same schools together, all that stuff. I literally tripped over family everywhere I went, and absolutely loved it all growing up. Of course, I could have done without babysitting for them, having other little hands touch and break my stuff, all the things little kids fight over.
On the other hand, I have decided to only have two. While I do have siblings, neither of my two brothers have any children, and my sister lives 4 hours away, so it's not like we see her and her boys all the time. My sons will not grow up in the same environment I did, and I'm okay with that, even though I know how much I enjoyed it. I want to take them everywhere, and show them the world, and give them all that I possibly can in terms of time, attention, and resources. The more children I have, the further those things get divided up. Not everyone looks at family size that way, but it's my perspective on it.
Do you and your husband have lots of friends, and a rich and full life outside of your family bonds? If you were to find yourself without those people, would you still have a place in your community where you felt appreciated and loved? Many people in this world do not have large families, but build their own through their social networks. You can look at that as the family you choose, not necessarily the one you get stuck with. Please keep in mind that being related doesn't mean you have to like each other. My Mom is one of 6, and none of them talk to all the others. They pick and choose who to stay close to, and who to avoid. It's a sad situation, and I wonder if there were only 2 or 3 of them if they would be closer, because that's all they had.
This is a hard decision to make. There are no guarantees either way, and both choices have their ups and downs. It really comes down to what you and your husband view as a "complete" family.
Best of luck.
Jess