M.H.
Don't be too worried. All kids are different. My eldest son was always in the 95th -100th percentile and I worried about him being too big, then my youngest son was in 5th-10th percentile and I worried about him being too small. All kids are different and averages are a big range. My almost four year old still fits in clothes for two year olds. I have accepted that he is on the smaller end of the scale and that is just fine. He has a friend who just turned three who is bigger than him, it's just how it is.
Your son may not have eaten much that day because he didn't feel well so your mom and grandma may be thinking he's so small because he doesn't eat well, they're putting 2 and 2 together and getting 5. You know your son and know if he usually eats well or not, trust your instincts and don't think too much on your grandmas comments. People of that era do tend to want to see "some meat on the bones". Don't let her comments scare you, some kids are just bigger than others, that's all there is to it.