D.B.
Regardless of her behavior, he shouldn't be hanging out at your place. You are not together and this infringes on your personal space. Moreover, she needs to learn that she has 2 homes and 2 competent parents, 2 places where she is love. She needs to see her parents as heads of separate households. His constant hanging out at your place is going to confuse her.
Do you think he's hanging out at your house specifically to undermine you as a parent? To make sure you have no other relationships? To show you what a cool dad he is? To get back with you?
If he picks her up for HIS time, he leaves. Have her bag packed and your keys in your hand so you are all leaving together and not hanging around at your house. You have your time, and he has his. If you can drop her off at his place and pick her up, that's good too although it is an imposition. Otherwise have him pick her up at your place (with you headed out), and then you pick her up at his place at the end of the visitation time to bring her back to your house. That way there is no prolonged time at your house and you break this habit.
If you find that he's not around when you are picking her up, then you will know he's more into controlling you and making you jump through hoops.
And all the special toys he buys her can stay at his house so she can "visit" them the next time. If he gets her hyped up on candy, he'll have to deal with it. The candy doesn't come into your house. Don't engage in conversation - he's responsible for her on his time, you're responsible on your time.
I'd stop talking to him about it (which doesn't work) and just start taking action. You can't control him, you can only control your own actions. If it doesn't work (like he's not there at pick-up time) and you have to have the attorneys get involved, it can be expensive. Parental alienation is taboo in legal circles - and that's what he's doing if he badmouths you to her and says you're the mean mommy or whatever.
But you are under no obligation to entertain him at your home.