You have a great plan. Just go and have fun and don't feel guilty about it.
I am sure your husband can manage and look at the bonding time it will give hubby and your son... wonderful! Your husband is obviously on board with this as he is planning his special dad time with his son.
If you build this up like it is going to cause strife.....your 4 yr old will pick up on this, then your 4 yr old AND your hubby will feel like you don't believe in them to manage on their own.
Enjoy yourself, come back recharged, you will feel better, you will be a more relaxed mom and wife as well.
Look at this as you are going on a mom adventure and your son and hubby are having an adventure on their own.