" I said well seeing there are some odd jobs to do around the house and no kids or wife to drive you crazy maybe you could get some house stuff done. And left it at that."
You said it yourself-- you left it at that. He did what he felt needed doing. If you wanted something specific done, that was the time to say something. So, it's sort of like "I wanted him to do stuff, I asked him to do stuff, but he didn't read my mind and so he did the wrong stuff."
I'm sorry, but you sound incredibly ungrateful and pretty petty about this. My husband handles most of the finances too-- it's his choice and he is our breadwinner-- so, I'd be a bit surprised if something similar happened, but you know what? I *trust* him. He works really hard for us and if he wants to make an independent decision, I'm cool with it. Furthermore, I would never have even asked him to "do" any work while I was gone with the kid. Why? Because the guy works hard, daily. If and when he has a few days to himself, I want him to enjoy it, not spend it working on ambiguously-identified tasks. (Yes, we do have an old house filled with plenty of projects remaining, but I want him to be able to relax, watch Lord of the Rings for the umpteenth time, have chill time, maybe meet up with his friends for dinner or a drink or go to a game.... see where I'm going with this? I find it far more strange that one feels entitled to tell another person what to do with their rare 'free' time than I do for a husband to want to do something he feels is nice for their whole family.
Oh, and my husband takes Kiddo off for a few days every so often; at least once a year they are gone for a week together. He's NEVER asked me to do housework/house projects while he's gone. NEVER. I sometimes choose to save some things for when they are out, like doing the Marmoleum sealing, or this weekend, they are gone and I'll be using that time to spread out my work (making curtains for some neighbors). That said, those are my choices to do tasks at a time when I have more space/less interruptions. Because I want to do those things, not because he expects it. You might consider that the next time an occasion like this comes up. Sometimes, less is more.