This is a tough situation, but it's pretty common. I feel this way from time to time, as a SAHM. My sister has it worse--she has five kids, and her husband is nagging her to go back to work full-time. The youngest is not even in school yet, and her husband said not only does her want her back to work ft, but he is not willing to pick up any of the slack, or hire anyone, whether it's to clean the house or be there when the kids get home! Yikes.
Here's what my husband and I have been doing, and so far it's better than it was...
We got three bluebird cards--they are prepaid cards that ring up just like credit cards (but prepaid). One card is for household expenses, including groceries. The other two cards are for us--each of us uses one for personal spending, allowance, etc., and we each get the same amount ($50/week each). We are tweaking the amounts we put in the house account, as we see what our expenditures are. It's helping us get our spending under control, and manageable. Also, I think having some personal spending money, and the same amount, helps us to feel as though we're on the same team, working together on it.
Do you think your problem might be two-fold? What I mean by that is, I think every husband of every SAHM wife, at one time or another, freaks out over money, and also (maybe at the same time, but maybe not), wonders "what does she do all day?" Because for most dads, when they stay home with the kids, it's a different scenario, at least with most of my friends. When they are with the kids, that's their only responsibility. When YOU are with the kids, you are also cooking, cleaning, doing laundry, etc., etc. I love my husband, but when he takes the kids, I typically come home to a messy house, and nothing has been "done." Therefore, I think a lot of guys think that it's pretty easy, this kid-watching gig. If you think that's an issue, I would maybe take a day or two, and write down everything you do. It will be a pain, but it will maybe help him to understand--be specific, and write in time-frames, like 8:15-8:30--cooked and served breakfast. And save every receipt from your weekly spending, so he can see how much groceries cost, how much that trip to the post office, home depot, and the dry cleaners cost. Either that, or take some time off--a weekend away maybe, WITH a to-do list. Not that you're trying to keep him busy, but tell him there are things that need to get done to keep the household running. That way, he's not just "watching the kids," but getting a sense of what you do that goes unnoticed. I don't know if any of this helps you, and sorry if I rambled on! Good luck--you'll get through this.