I agree with Suz T. don't give in to him. If he kicks or hits you tell him to STOP IT NOW and sit him down. Then go back to your conversation. The more attention you give him the more he'll keep doing it.
Talk to him now about how to behave when you are talking with someone. Teach him to say "Excuse me" and then wait. That you like to visit with your friends just like he does. And that it is not acceptable to interrupt or cause a scene.
Bottom line is, if he doesn't want to listen to you "actions speak louder than words". Show him you are boss by not giving into his antics.
You may have to explain to your friend that he's going through a rough phase, and to please just ignore him.
Practice at home when you're on the phone, talking with your husband, etc. DO NOT answer your son when he interrupts you or is rude.
You might even try modelling his bad behavior with him sometime, pull on his clothes, "scream" in his face and then say, "You don't like that do you? Mommy isn't being nice is she? Well this is what you do to me. And I don't like it either." 4 should be old enough to understand simple role playing. Then ask him what he thinks he should do...and make a game of it.
Best wishes