Of course it's not normal for coworkers. You are in shock and he has you doubting yourself. No one I know is OK with a spouse lying and hanging out in opposite sex coworker's homes alone at night watching TV. He is guilty of spending inappropriate time with a female coworker multiple times and lying about it. Have they actually had sex? Without video proof you don't know, but he is guilty of being 100% wrong already. Best case scenario if nothing happened he's acting like a selfish, hurtful teenager with no respect for his wife.
You need to try to remain calm-as in not defensive or frantic-and demand that he handles this with you and with her. "Nothing happened" is not good enough, because something is happening, he's lying and doing something he shouldn't. Does he plan to keep doing it? Or does he plan to quit seeing this woman even if it means quitting his job?
I'd talk to someone who knows him about what your most effective first step should be based on his personality. But sounds like you need to tell him to quit seeing that woman (and she's a huge creep btw for hanging around someone's husband like this) and see if he complies and how he acts.
He does not sound sorry though, or like he has any clue how to behave like a married man, so I'd be ready to speak to a lawyer. It is definitely time for an ultimatum. You or her.
If he is REMORSEFUL and honest, and stays away from the woman and does his best to make it up to you and never acts shady again, then counseling would be a good first step if there is hope.