Hi, S., Congratulations on the blessing of a new child in your family.
It sounds to me like your husband is overwhelmed. From your "little about me" section, I take it that you used to work outside the home since you say your are "now staying at home." I'm assuming that you used to bring in some money to the home. So now your husband is the only breadwinner -- that in itself is very stressful for him.
Add to that his change at work, which is also stressful -- learning a new role, wondering whether his job is secure, etc.
Add to that your new bundle of joy (and spitting up and crying...), which is stressful in itself, and I think your husband needs a way to relax. Often when we have a new child, the spouse who stays home looks ahead desparately to when the other spouse arrives home. We look to that person for companionship (an adult to talk to...hurray!), and, as well, we want a break from being the sole caregiver all day. So the poor spouse opens the door to more "work" -- no time to take a break, relax from the day, and then be ready to take on more purposeful activity and helpfulness.
So! If I am right in my assumptions above, I would suggest that your husband schedule in some break time for himself after work. A walk, a short round of golf, a visit to the gym, a jog, hanging out with his friends, alone time, reading, thinking...whatever works for him BEFORE he takes on the additional work of being with your son (and you). This is especially important if he prefers Introversion (gets his energy by going within and/or being alone). Otherwise he will have little energy to be fully present with you and his son. The result of this will be the symptoms you describe -- frustration, annoyance, impatience.
I know you need a break too, so perhaps you could have someone come in to help you -- a friend of relative or neighbor, who will give you some time to do the above things too (whatever helps you relax and re-energize), so that you have a break and can relax away from your son. If you prefer Extraversion (get your energy from being around people and in the outside world), when your husband comes home, you will want to immediately start talking to him about your day and wanting attention and responsiveness from him. If he needs some down time, he will not be able to meet your needs (nor his son's).
So, my bottom line -- I think it will help to pay particular attention to both of your energy, and to do whatever it takes to have enough energy to be fully present to each other and to your son.
Then both of you will have the patience to be well with your son and with each other. Hope this is helpful.
E.
____@____.com