Dear A.,
My son is 3.5, and we just embarked on the potty training train this past September. I can report that now 4 months later, I still carry extra clothes and my son still has 1-2 accidents a week, but he's doing well and he's learned a lot about his body and when to tell that the pee or poop is coming. I was surprised just how long this awareness took to develop, and I admit in the early days I was a)anxious about public accidents like your husband (he did pee on the bus once, which was embarrassing and difficult to deal with) and b) less patient with him than I wish I'd been. I thought he was misbehaving initially when he had accidents (ie. willfully refusing to use the potty), because I didn't understand how he could be so good at it sometimes and so bad at other times. A friend with older kids helped me to view it as a skill to be developed, and we whenever he had an accident, we would say "that's okay, you're still learning." He quickly picked up on this phrase, and it built his confidence and helped him relax.
I absolutely agree that once you go down the potty training road you've got to do it all the way (although we do have G wear pull-ups at night). Wearing a diaper out of the house I think would be confusing. And with a little practice, you can get good at dealing with the accidents. Also, there is a fabulous folding portable potty that we got and took everywhere with us - it has disposable plastic bags that fit into a small seat. And we have a kit with spare underwear, pants, socks and wipes that we take with us wherever we go too. We've also learned to interpret what is just wiggling and what is a "the pee is coming" dance. All in all, I can count on one hand the number of difficult situations we've been in with public peeing/pooping - it's really not that bad. So, tell your husband to be brave and be patient. Soon your son will be using the toilet all the time and your worries will be behind you!
Good luck.
C.