I am assuming this is public school, and frankly I have found them to be defensive and difficult at times. It took me 2 months at one point, to get my daughter in to a different class that made all the difference in the world to her development. That was 6th grade. But I had to go to the principal and even threatened to go over her head if it wasn't solved immediately. If I had not done that she would have been in lower math all the way through middle school.
But, I also didn't do some things for my son that I now believe in hindsight, that I should have, and I think that he had a more difficult time in high school because of it.
My children are now 20 and 24, I've been through the system and I can tell you in that hindsight, that I wish I had even been more assertive earlier in a couple of instances. It would have been better for them emotionally if I had stuck up for them instead of the teacher, or because of my concern to "make waves".
With how your son feels and is behaving it's a pivotal point that could very well impact him longer than you think. Don't worry about his teacher's feelings; only your son's. If he is experiencing that level of anxiety he NEEDS YOU to be his advocate.
You need to DEMAND what you want. Call and request an appt with that resource teacher. That resource teacher has no right to label your son. Don't let him accept the label she's pushing on him. And let her see that you will not put up with it. Absolutely insist that she does not label your son again. She should have to look him in the eye and tell him he is not stupid; he is not ADD, so your son can see that and regain his confidence.
In fact, if she is truly a good well-meaning teacher, she may not even realize how she hurt him and will want to help with damage control. If she doesn't, then maybe you should get him away from her anyway.
Ask for a separate appt with the IEP counselor, so you clearly understand what the IEP report says and means for your son's development program. And if you don't get satisfaction, go to the principal or even beyond.
Children are 'tracked' very early, so handle it now or it will follow him for his entire public school career. If he needs more support than the puplic school can provide, then put him in a private school. Many have scholarships if finances are an issue.
My son has just been accepted to law school and my daughter goes to excellent private college too, and I believe that early intervention along the way helped so much.
I hope you heed my advice. Good luck!