She's absolutely old enough! I was changing my cat's litterbox by myself at age 7 or 8. However, your husband needs to change his approach. You or him need to sit down with her and show her step-by-step how to do it correctly. Help her with it a few times. Then continue to monitor to make sure she gets it right. She needs instruction and patience until she learns how. Then again, I grew up on a farm so by age 7 I was mucking out the cow and horse stalls (yes, a 7 year old with a pitchfork! *gasp*) and responsible for feeding all sorts of animals including our cats and dogs.
Our oldest is 5 and he has his list of chores (some daily, some weekly) like taking out the trash, making his bed, sweeping/mopping the kitchen and putting helping to load the dishwasher.
That said, there's no reason why you can't change the litter box. Yes, there's a >slight< miniscule chance of catching I-can't-remember-the-name-of-it. However... the only way to get it is if the cat gets it... the only way for a cat to contract it is to be outside. IF your cat goes outside sometimes, the parasite/bug/whatever needs to be present in his stool (cats get it from eating dead animals). IF it's present in his stool, you need to touch it with your hands within a few hours of it being into the litterbox. IF you touch it within that few hours, you then need to touch your mouth, nose or eyes before washing your hands. That's alot of IF's! :-) Seriously, just wash your hands well after changing it and you'll be fine. There are tons of other things that are much more dangerous that you probably do everyday.
You have a MUCH greater chance of getting it from gardening - especially in mulch where outdoor critters like to pee and poop. Make sure you wear garden gloves and wash your hands well after working outside in flower beds or veggie gardens. And make sure you toss the gloves in the laundry frequently. That is how most pregnant women contract it - not through the litterbox.