I would not think it would be due to the lack of milk. If he's eating well and not gaining weight, he absolutely should be screened for celiac disease -it's most common genetic disorder and is currently highly underdiagnosed. Celiac is an intolerance to the protein found in wheat, rye, barley and most oats. Here's a good overview of celiac (noting the broad range of symptoms):
http://digestive.niddk.nih.gov/ddiseases/pubs/celiac/
You don't have to have overt GI symptoms, the damage to the intestine can be partial and still result in malabsorption of nutrients. If your doctor won't run the test, then you can have it done yourself at: www.enterolab.com (the lab was started by a top GI researcher that got frustrated with the slowness at which mainstream medicine diagnoses folks with this disease - only catching a fraction of the folks with the disorder and only one the disease had caused much more damage).
My son was similar to yours in that he ate well, but stopped growing for a period of time (between 8 and 10), but he was always very, very thin. So the pediatrician didn't worry - he just said he was going to be "petite". If I known then what I know now I would have insisted he be screened for celiac when he was much younger. He did, however, catch up height wise once he started on the gluten-free diet (growing 2 inches and gaining 10 lbs in 6 months and continuing at that rate for several years).