Why do you want to take her off milk?????
At this age she can have up to 24 ounces per day. She needs the fat in it plus all the other nutrients it has. If you want her to drink less anyway then offer her 4 oz. of 100% juice for a snack time drink. That is a whole complete serving of juice per day, 1/2 cup.
She can have lots of other fluids to drink like tea, water, koolaid, etc...I would make any drink mixes with less real sugar though. I make a pitcher of Koolaid and only use 1/2 - 2/3 cup of sugar. The kids love the tartness of it.
Kids at child care or preschool get milk with breakfast, milk at lunch, and often again at snack time both morning and afternoon. I did try to do juice one or other snack time during the day though. Then kids usually have milk again in the evening.
Four tippy cups are not very many for a whole day. My kids drink that much milk between the time they get home from school until the go to bed at night.