I hink it's really important that you try to get them into activities for themselves. I do not have a close relationship with my sister to this day (and I'm 38). We fought all the time and my mom MADE me play with her constantly. She was 2 years younger than me and I had to drag her everywhere I went. I hated it!!!!!!
If you can get them EACH into a SEPARATE activity that is "just for them" maybe it will eliminate that feeling your older kid has of not having anything for just himself (yes, I'm projecting my feelings!!!!!). But don't make all of them do the same activity either - just cuz it's easier for you to go only one place. That defeats the purpose.
Boys are boys too - so I'm not sure specifically what he will do with his brother that he won't do with his sister, but sometimes it may be that it's because it's a brother thing (I have 3 of those too and he doesn't want to do things with a girl!!!!!).
I would say when they are mean, it's time to separate them and they play by themselves. Explain to your daughter that "timmy" doesn't have to play with her ALL the time and allow your son to say no occassionally and have it be ok. By then turning around and playing with your son I would say he is making a statement on wanting to have a say in who he plays with instead of being forced to play with his sister. I would also tell your son that sometimes we need to play with our siblings!