Hi M.,
This sounds so familiar...
Unfortunately, I was the same little girl as your 4 yr old daughter, and it breaks my heart. My father was really only a "father" because the court forced him to be one financially. I am sure he thought that was good enough. But I was truly blessed with a loving, strong, and selfless mother. Although she and my father never got along, you would never have known her dislike. Never for one minute did she tell me anything bad about him, so you are already doing the right thing in that regard. However, you have to keep your distance from him at work. My mother and father also worked in the same building, but just treated each other cordially when they came across each other. I know its hard to not go off the handle with him, but this does no good, as you obviously can't force him to do anything. Just try to avoid him completely while the court case is pending.
If the court case comes out and he doesn't want visitation, it will be a very sad adjustment for your daughter, and there is no real way to explain the situation.
I hope that once personal issues between he and you are over, he will want to be a father to her. It will be very confusing for her if he decides not to be a part of her life. The only advice I can give is to do your best to make her feel confident, secure, and loved a thousand times over. When I used to ask, my mom would say- "Your daddy loves you very much. He just isn't able to come and see you right now, but I know how much he misses you." One day, he will have to come to terms with his actions. I am 33 and my father has not spoken a word to me for 6 years and has never met his grandchild. Unfortunately, some men never get it, but there isn't anything you can do about that. But I always felt loved and safe because of my wonderful mother. You sound a lot like her! Stay positive, and she will be just fine. :)