You asked in front of the whole group to speak to him privately, so everyone in the group knows that there was an issue between you. You then left swiftly and clearly unhappily so that also lets the whole group know that you and he had an issue even though the others may not know precisely what it was. Prepare yourself for someone else in the group to ask what was wrong, etc., and prepare what you plan to say if that happens.
Also be aware that next time the group meets, he (if he has any sense at all) may not be there if he's decent enough to be embarrassed, and that should be the end of it. But if he's a complete idiot, and he may be, and turns up again, he either may be overly nice to "make it up to you" or he may be cold because he knows the whole group is aware he somehow upset you.
I'm just laying out these possibilities so that you can think through them before you meet with the group again; you do not want to walk into the next meeting or dinner and be surprised if he is very chilly, or his wife is chilly, or if he's excessively nice and attentive and complimentary. Just be aware that the whole group knows there was an issue so you might encounter some additional response to this situation, and be ready for it so you're not caught off guard and get upset again.
I think you handled it the best you could in the circumstances and you did indeed call him out, just not in front of others. If he is around again and makes another comment, I'd turn to him -- not go aside separately -- and just say very calmly and condescendingly, "Bill, I think we covered these kinds of comments at the restaurant that time, didn't we," with a withering look as if he's a clod too thick-headed to remember that he already got told off and you have to remind him in front of others. I would advise against shouting it; just be the very cool and collected adult treating him like a child who is too socially dumb to remember that He Is a Clod.
A lot here depends on how well you know his wife and how you and she get along, I think. If she knows he's an idiot in this way, she might just be pretty upset about it, but if he's the type to tell the story in a way that makes him look better and you look "over-sensitive" etc., she might not know what to make of it. If you know her really well and are good, real friends and not just book club pals, you might be able to tell her why you called him aside. I hope she'd be appalled.
I agree with the person below who said it's time to boot the husbands from book club. Are they really into the books and participating, or do they just want to tag along for the social side of things and turn up only when the meeting's over and the snacks come out, etc.? It's a tough call, because if some husbands really do want to engage in the book side of things, you don't want to tell them to take a hike, but if they all are mostly there for the socializing, it's time to start meeting when they can't be around. That might not work if some of the women members want the husbands there.
I would just think through how to respond to different results here (he's cold, he's gushy, he told a different story about what you said to him) and have those responses ready, and then just wait and see if anything else does happen. It may blow over with nothing else said and even with him starting to back off coming to the group. Just be cool and as unemotional as you can, and take the high road of "He's just not aware that even small comments hurt -- poor dear."