L., she'll more than likely grow out of that. I have 4 kids and one of mine was like that when he was really young like yours....just wanted me and not daddy at all. I also had a child that only wanted his dad and not me at all, while dad was around.
I have another child that wouldn't even look at men or get near them, except her daddy, but all woman were okay. She is almost 4 and has grown out of that. My first child let anyone and everyone hold him. My 2nd child preferred his dad over me and wouldn't even hug his grandparents until he was a bit older. Each child just has their own needs/wants/personalities/oddities.
Of course right now, you would never know that my oldest 3 were like that when they were young. They will talk to anyone and everyone. lol
Right now at her age, she is really just needing you. I hope that you and your husband are not reading into it too much. Some kids are just that way for a bit while they are young.
While you are gone with your work 4 days a week, even though she does fine at the babysitters, she is needing you. So when you are home, just hold her as much as you can. She needs you and that is the only way little ones can communicate what they need. If it wasn't necessary, she wouldn't be crying. You want to build a mentally healthy, loving child. Little ones at that age can only tell you by crying, etc.
I had read this the other day and thought it fit well here...especially for those that think what is going on is wrong... "Why are babies treated as less than human? Why would we WANT to teach our children that we won't be there for them? Why should we get to pick and choose which needs are "real" and which ones are not." (Anyway, this was mainly talking about letting children 'cry it out'.)
I would just give it some time. Your husband just needs to wait until she needs him. Let your baby be the one to come to him when she is ready. {{HUGS}}
Mother of 4 well adjusted, happy, kind, compassionate children