J.J.
This is a very difficult problem because your kids are old enough to realize something has changed and not in a way they want a change. My son is only 2 and me and his father has been split since just after his birth. My sister on the other hand has a 6 and 13 year old that deals with their father being out of the house. The six year old is mad and blamed me for his parents break up. I told him that it was not my fault and that it was between his mother and father. Sometimes adults can not live in the same house and he should not worry about that because it is an adult problem. I also told him that he needs not worry about who's fault it is just know that his mother loves him, his father loves him and he loves both of them as well. Always make sure they focus on the fact that they are loved and just because Daddy does not live with you anymore does not mean that either of you love them any less.
In both my situation and my sisters situation we never talk negatively in front of the children about their father (try not to) and focus on the fact that their fathers love them dearly. It makes it harder when the father is not consistent with seeing his children. My sons father is like clock work three days a week but my neice and nephew see there father once a week if they are lucky and he tends to take his son more than his daughter as well. Consistently does not mean that plans/days of visitation can not be flexible. There are times that it fits better for me and my ex to switch days or times for him to spend with our son and we work together as much as possible to do just that. Our families also both respect our wishes of not putting bad thoughts about anyone (either parent) into his mind. It is always best for the kids to receive as much love as possible from as many people as possible. When my ex's family calls and wants to take my son because they haven't seen him enough in the previous week or two I try to accomodate them as much as possible.
The most important is not what you say is the point I am trying to get across but your actions with your ex and his family. Do not be afraid to set ground rules of consistentsy with visitation. It is so important because if he is not consistent your 3 little angels they may sometimes feel they are not loved by their father and then you have something else to explain to them and I think that one is far worse to have to explain then why is Daddy not living with us anymore.