I agree with the posters that this sounds like a water issue. :( Hate water issues!!
When you have a company that cleans out ducts, they come in and drill holes in places in your ductwork, and put high pressure air through them to blow what's inside them into a container of sorts. The container has a big filter in it. They will identify every return in your house and use it as well.
It cost me over $700 to do my house. It was over 20 years old and had never been done. There was a lot of "stuff" in the vents, including dog hair from the previous owners' pets. I'm glad I got it done, but it won't need to be done again for a very long time.
I would say that it's worth doing if your hubby is having allergy problems, but this musty smell is most probably NOT coming from your vents. You do need to figure it out, though. The worst thing that can happen is to start seeing mold - then it's the tip of the iceberg and you end up pulling out an entire room! (That happened to my mom when the water heater leaked because of improper installation. Bad stuff!!!)
Dawn