It's quite natural for people to compare. But they shouldn't do it in an arrogant way or suggest that there is something wrong with one child or another. I TOTALLY agree with you. It's very irritating when people take the milestones too seriously.
My perspective isn't just about families and friends. So I'll share that I get this all the time in my in-home daycare. Here are a few scenarios...
One little girl, with me from 8 weeks to 2.5 years was great in every way. She was learning, happy, enjoyed coming, but her parents decided that we weren't potty training her fast enough. So they took off to find another provider that would promise to let her wear real underwear come what may, pee on her furniture and all.
The next child, more recently left for several reasons. Ultimately the excuse was money. But in the weeks leading up to the parents extended lay off, she had been sharing with me that family was pushing her to put the child in a center. I had this child from 5 months to 3 years. She was mostly potty trained, more so at my house than at home. She was learning to talk well and communicate here, but again, not so much at home. She was shy and didn't care much for other children. She was more of a sit in a grown-ups lap kind of child. But she was coming out of her shell. So she gets yanked from a place she was perfectly happy with because her cousins were supposedly doing EVERYTHING so much faster.
Another scenario that I have dealt with over and over is that children don't always show their parents what they know. I am sure this could be true in the opposite direction too. I know it can. I have children that refuse to talk to me and will act like they aren't getting much out of circle time or phonics lessons. But then they go home and display all kinds of new words, spelling success etc. for mom and dad. I had a little girl for over 2 years. She was learning to read at a wicked fast rate. I was excited and I KNOW I shared this with her mother a long the way. Well low and behold, the mom took off to put the child in a BIG preschool. Then a few weeks later she was all excited and told me that her child was reading. She was all excited and like... "SEE, she needed a preschool". Um no...she learned to read in my house. Where have you been?!
On one hand I think you should be able to say, just back off. On the other hand, make sure you aren't being too sensitive. Sometimes people just don't say things in the best way. I know I'm very guilty of saying things in a way that I didn't mean at all. We all do it sometimes don't we?