With issue with my own sister I can tell you that being honest, open and direct are the best way to approach. The worst problems are when either part do not know the whole story, and that is in this particular case what happened. He does not know that you were going to offer him to tell the kids.
Little things can go a long way. Before the congrats on your pregnancy I would call him and say... I just want you to know that I love you... if he does not answer leave that message in his voice mail. This little gesture will open the door, I am sure of it.
Then you can send the card and days later a little gift or something will be super nice.
Some times it is difficult to talk in some cases it's impossible to talk without opening a pandora box and is very difficult to come to an understanding or agreement, even you get the things out in the open but the feelings and emotions are there.
You can start a healing process in the relationship with actions and nice gestures... a short and steady pace of good will and good things towards them, meaning your brother and his wife and they will notice and soon will be reciprocated.
I understand your concern about not leaving your kids alone with him and yes you should always think and protect your kids first, that is the single most important thing in the world...but as said, he can still have a relationship with them and with you.
For some time you may have to tip toe in the relationship, specially with such sensitive things as the pregnancy but it is also a wonderful opportunity, I guess he wanted to see you all happy and excited for him and for the new baby and all he got was a very "cold" is too early in the pregnancy... to be honest that would have bothered me too, even if that is not what you meant and intended, he does not know what you were thinking.
Do towards him what you would want for yourself, before any meaningful action think first if you would like it and if the answer is no, then do not do it or say it.... Think the little details that you really appreciated from friends and family and then think what you can do for them that will tell him. I love you and I am happy for this baby.
The next time you see them, have a little something, mabye a pijama... and say I was shopping and saw this, loved it and could not help myself to buy it for MY new baby!
Go to the store and buy the most helpful things that a newborn will need. (I remember that a neighbor gave me a bunch of things like the nose aspirator with the drops, the drops for the upset tummy) you get the idea, the toilettries for the baby.... give them in other occassion, not the same day.
Another day get them one pack a of new born diapers and a pack of the following size.... this will tell the that you are excited and happy for this baby!
No thing have to be expensive just well thought of and they will go a long way.
Hope this helps! Good luck!