A lot of people have offered great advice, I would just be adding more of the same.
But, I wanted to share a personal observation to help you:
You mentioned that you both have full-time jobs, but not always money in the bank. Unless your full-time jobs only pay $10/hour, you should be able to pay bills & provide necessities for yourselves without problems each month.
If you cannot, then you are either 1. Living outside your means (mortgage is too high, car payments for a brand-new high end car, etc.) or 2. You are spending money on wants as well as needs.
Prior to this year, my husband & I should have been living VERY comfortably. We had a home with an affordable mortgage (1/5 of our monthly take-home pay), we are conservative with our energy & water, & we don't carry credit card debt, nor have student loans or car payments.
However, because we had money, we spent money. A LOT of money. Going out to eat instead of making dinners/lunches @ home. Every day. See it, want it, buy it. Lots of money on equipment for hobbies that we didn't pursue long term. You get the picture...
In January, I quit my good paying job, & went back to school full-time. Which cut our income in HALF. So we tightened the belt. No more going out. Make a list before going shopping. Engage in the activities we already have supplies for. Etc.
Imagine my surprise when I checked our bank account last month, & found out we had a couple of THOUSAND dollars excess, while paying our bills!!
Turns out, we didn't need to spend the money we had been, at the level we were spending. We chose to do so, because we thought it was fulfilling our needs. But instead, it was just draining our ability to save, & afford more high-end, long term items.
There are families of 4 who get by on 35K a year. With a roof over their heads in a decent neighborhood, food on the table every night, & the ability to participate in hobbies as a family.
So a lot of managing your money, when you have good jobs & easily enough funds to afford your necessities, is not so much "budgeting" as it is making the decisions between NEED and WANT.
Cable is a WANT. Cell phones (in today's day & age) are more of a NEED, but not necessarily the high-end smartphone packages.
Cars are a NEED in most cases, but a 5-yr old paid in full car will usually do just as well as a brand new, hundreds-each-month in payments car.
Food is a NEED, but going out to eat more than once or twice a month (for date night, because keeping the romance in your relationship is a NEED), is a WANT.
See the difference?
And I KNOW, it's really hard to see the differences until you are forced to sometimes, because you technically probably have the money to continue to satisfy your wants... but think long-term. Babies? Bigger house? Minivan? Kitchen upgrade? Vacations? Make your wants more worthwhile, long-lasting & beneficial to yourselves overall, & sacrificing the short-term wants won't be as hard.
So when you go through your bank account statements for the last 6 months, tally up the expenditures in buckets (utilities, groceries, gas, dining out, other entertainment, credit card/car/student loan payments, etc) & see where the biggest chunks are hitting, & what you can scale back.
And then, make one long-term goal together, that you can work towards, so that you feel good about eliminating some of your current wants.
Paws crossed that you & your husband can have an open dialogue & come to agreement on what needs to be done!! T.