I wouldn't worry about offending them. They are being rude and need to be told to stop offending you.
I want to comment on the speech delay. Your son sounds similar to my grandson who was still not talking at his check up at 32 or so months. His pediatrician suggested that he could just be slow in learning but that an evaluation would be helpful. She didn't push it at all and I wish she had much earlier.
School districts are mandated by Federal law to provide services to children who have difficulties that might prevent them from success in school. The evaluation is free and so are the services after the evaluation.
My grandson started treatment just 2 months from his 3rd birthday. The program that gave him one on one treatment ends at age 3. He then went into a preschool program at Headstart but he was one of 20 kids and the only one with a speech problem. He saw a speech therapist a couple of hours a week.
Because he didn't adjust to this classroom and took his anger out on everyone around him, hitting, pushing, crying, refusing to cooperate with classroom routines the district placed him in a therapeutic preschool that maybe has 8-10 students and 3 teachers. He still doesn't get very much speech therapy but this is helping him learn how to succeed in a group. He has blossomed in many ways.
But his speech is still nearly unintelligible tho better. He'll be 5 the end of June. Because he now is covered by his father's insurance he is starting with increased speech therapy at Kaiser Permanente. If he hadn't had insurance the school district would have found a way to increase speech therapy.
Because he has both speech and behavior difficulties he will be going to a special kindergarten this fall.
All of this has been done free with the exception now of insurance co-pays. And the district also provided support for my daughter as she learned how to help her son learn.
Yes, your son may be delayed and be able to catch up without professional care but it is even more likely that he needs speech therapy, hearing improvement or ??? I strongly advise that you make an appointment with your school district's Intermediate Education Services for an evaluation.
If he doesn't need intervention or even just a little you'll be relieved. If he does need intervention, as my grandson did, you will be helping him to not only fit in with other kids but also make it possible for him to succeed in school. If he needs treatment the better it is for him the earlier you start.
This process has been painful, especially for my daughter. HOwever, it would be more painful if he's not able to work independently at school or has to delay starting school.
BAck to the rude mothers. Deal with them in a respectfull manner but don't let them continue to do this to your son. Perhaps mostly they just need education.