Ugh..
Divorce is always so awful. Each parent has an expectation and of course it is not the same for divorced couples.. Remember, this is why you are no longer married to him.
You say you have an agreement about notifying each other about babysitting, but is this just a verbal agreement or actually part of the divorce agreement? If it is verbal, it obviously does not mean anything to your ex. You could document these occasions if it tends to happen a lot.. Meaning he leaves your son with sitters many times a year with a sitter instead of watching son himself.
HE is the one making bad choices. HE is the one that will have to explain this to your son in the long run.
YOU cannot control this, instead you could suggest options.As the father he has just as many rights as you have to make these choices.. Not that they are correct choices, but it is just part of it.
Even if you were still married, there would be times, when you all would have gone on a trip and son would have been left with family or friends.
This 1 time of both of you being out of town at the same time could be fine, if as suggested some grandparents could step in or a beloved friend of the family watched him.
I TOTALLY understand how you feel. I do not understand how some parents only see their children once a month for such a short time and yet, seem to take it for granted, by not thinking about what impact their silly decisions can have on their children.
Your son will grow up and without any help from you, he will realize his father is a jerk. TRY, TRY to not be a part of this.. let son find out you always tried to make the right choices while his dad had totally different priorities. ,
Breath... follow your heart..