J.F.
I whole-heartedly agree with what Kathi H said in her response. Although, like the other moms said, you also really need to make sure there isn't more cause for concern by looking into it.
When my husband had to take his ex to court because he wasn't getting visitation he deserved and she was constantly playing games, we had a lot of great conversations with the law guardian we all had to meet with. In our case this hasn't really happened, but one point he made was that he sees many children act up because that's what they think the parent wants to hear. If she said she doesn't want to go, maybe you're showing signs (that you might not even realize you're doing/saying) showing her you'd rather her not go. Children just want to make their parents happy.
I'd also suggest talking to her (she should definitely be old enough at this point) and look into the situation more yourself if possible. Talk to her father (if you're on even semi-good terms) and explain your concerns in a calm, unaccusing way. Maybe he has an idea of why this is happening.
It also sounds like he's not been around much, so obviously she's much more comfortable with you than him. She may just not be used to him and the way that his home runs compared you yours. Maybe she just needs some help in the adjustment between lifestyles.
Hope that helps a bit. Good luck!!