I had 3 children when we lived stateside and I stayed at home (though gas and food wasn't as expensive as it is now). We've lived overseas for 3 years (now have 6 children)
and are due to head back to the states in a year. I am already dreading the loss of COLA.
One thing you can do to prep yourself is to put your COLA in savings for the remainder of your time overseas and just live off your husbands normal pay. Also, if you forsee a need to make a major purchase in the future (ie. purchasing a car, house, etc.), start making those payments to yourself now. When the time comes to make the purchase, hopefully you won't have to take out a loan and if you do it won't be for as much. (That interest really adds up!)
When we lived in the states I did a ton of garage saling. I bought clothes that my daughters and son would grow into over the next 3 or 4 years. When you need something is the worst time to buy it because you can't wait for a bargain. I literally clothed my children for about $50-$75 each year and their wardrobes are huge and trendy. Used childrens clothing is a dime a dozen especially in military communities where people are looking to unload of their unwanted goods before they PCS.
You can get some great financial advice from Dave Ramsey in his book A Total Money Makeover. He also has podcasts you can listen to online. He suggests having an envelope system where you have so much money alloted each month for specific things. For example all the money that you're going to use on fun goes in the envelope at the beginning of the month and when it's gone it's gone until next month.
He also suggests paying for everything with cash. We don't do that but I used to keep a small little sticky note on my credit card and everytime I charged something on my card I'd write the new total that I was going to owe on the sticky note. It really helped me to rethink purchases and if I really needed them that badly.
If you have any credit card debt, really work to pay that off ASAP. Once you're behind it's so hard to catch up. It's much easier to just stay on top of it. Never charge more than you will be able to pay off. Also, try to have an emergency fund set aside. If you have to rush home for a funeral or you have a major car repair you won't have to go in debt over it.
I hope something I've said is able to help. Good luck to you!
Edited to add a few more thoughts I had on staying home vs. working.
-If you stay at home would it work to be a 1 car family and if you worked would you need a second car?
-Would you need to pay for childcare? How much would you be bringing home after taking out that expense?
-Would you need to purchase and maintain a work wardrobe?
-If you really do need the extra money, is it possible to work from home writing articles for websites, selling on e-bay, doing some sort of transcribing etc?