Well, where are they making dinner? In the office? If they're making dinner at someone's home without any spouse present, I might wonder what's up, but if they're having an office dinner for some work purpose, it seems harmless, or perhaps other people are involved too, not just the two of them alone, but he didn't rattle down the list of other participants to you, or maybe they're not company employees. Maybe by "making dinner" they're referring to handling dinner, as in ordering/catering food -- coordinating dinner/taking care of dinner perhaps might have been a better description. I don't know, why not offer to assist if you're feeling uncomfortable about it all?
At the same time, I would think that if he had something to hide in terms of his relationship with this woman, he would not be telling you or inviting you, like he did, to the concert -- he'd just spend time with her and not say a word to you about it. Why tell you if he's up to no good and not wanting to get caught? He wouldn't want to flaunt an affair. I assume that is your concern -- that he is cheating, but as I said, a cheater would not want to even plant the seed of doubt in your mind, so mentioning events outside of work would not make sense. He would just claim he had to work late or something, and hide his activities. I don't think he would invite her husband or their parents as well if he is truly having an affair, that is incredibly brazen.
My boss has invited me to his daughter's theatrical performances before, despite him being married, while I am unattached. I didn't go, but he said his wife was asking why I hadn't come along, surprisingly. I didn't want her to feel odd about me being a third wheel but she didn't take it that way at all. He later invited me to come to a hockey game as his daughter was going to sing. Again, I didn't go, and again, he said his wife was disappointed. I don't know how long your husband and his assistant have been working together, but my boss and I have been working together about 10 years. We have weathered the recession and more together, whereas other members of our department jumped ship, but the two of us have been a constant, and his wife always sends him with something nice for me for Christmas. On occasion, she has baked cupcakes for the office. Some people are just genuinely friendly, without there being ulterior motives of an affair or romantic interest, and some bosses these days still DO appreciate employee loyalty.
If he hasn't shown you a reason to distrust him, I would not blow up at him, assume he is cheating, or push him away. Doing so, might make things worse. As someone else suggested, spend more time with him, alone, if you feel that your marriage is threatened, or that you have neglected him in any way. As to some people's mentions of other employees getting wind of things and becoming upset, I don't think this would be an issue, especially if your office has separate departments. Some of the bosses here take their assistants out for lunch, some of them go to happy hour. Some have met up on the weekends to catch a movie along with their significant others, and I wasn't invited. I would not expect to be invited along if I don't work for their team or department. If I were a support member of your husband's assistant (meaning we're in the same department and I might overhear the invitation), I might feel left out, but even then, a lot of people don't bother inviting people who have small children, a spouse, or an elderly family member that may be needing them home immediately after work because they know the answer will be no.