Maybe a little bit of both. I've experienced this with my Son. He has some delays and has Sensory Integrations Disorder. He's getting therapy (OT) and play therapy and we are getting counseling to help us enforce the proper consequences for him.
Some kids time out doesn't work. Our Son doesn't get it. If you take away toys or videos, he gets that and it works better as a consequence. If he can't go outside or to the park/mall, he understands that.
Get creative and come up with some other consequences and if that doesn't work. There could be other issues you don't know about. You could get him evaluated, play therapy or counseling for you and your husband as Parents.
We use Love and Logic and Parenting by Design techniques. You might want to try some tips from these programs.
I've had a hard time coping. At times my Son gets aggressive and I need a time out. So, go away for a few minutes and come back. I go and pray and it really helps to cool down first. Feel free to email me and we can vent with each other. I'll be glad to give you my email address or phone. Try journaling that has helped me in the past. Things are getting better for us, now that we are getting to the root of the problem.
You could read some parenting books,etc...