What is your response when he starts the barking? If you give him any kind of attention trying to "calm him down" then you are rewarding the behavior.
Have you ever watched Cesar Milan (The Dog Whisperer on National Geographic channel)?
He deals with this sort of thing a lot on his show. He often will use his index finger to sort of poke them in the chest (just to get their attention via touch- not to hit them!) and simultaneously "shush" them. He uses this sound that I don't know how to type phonetically, lol... sort of a "tscht" sound. Then, when he stops barking you can give him a command (like sit) and then praise him for sitting.
Most of the time it seems like annoying dog behaviors are associated with lack of exercise and attention in general. Do you take your dog for daily walks? Do you walk him correctly (YOU being in charge of it and him being the follower, not the leader)?
My dog gets very excited when my husband comes home and starts barking and whining, etc... because she has been accidentally "trained" to "find" the kids for him. They hide in the house when his car pulls into the garage and when he comes into the house, he "looks" for them... she LOVES this game!! So she gets very excited when he comes home and she gets to play it. When the kids are in bed, she doesn't do any of the whining and barking, because she is conditioned to find them "hiding" when he comes in. If they are in bed asleep, she knows there is no game, and no "reward" for "finding" them, so she doesn't get all wound up.
So, you have to figure out what the "reward" is for all your dog's barking... Do you pet him and show him attention and tell him that "it's okay..." ? Then you are rewarding his antics. Figure out his reward, then get rid of it, and reward him for sitting and being well behaved (and quiet!).
If you can, try to watch a few episodes of the Dog Whisperer show and see if his techniques can help you out.
Good luck.