A.S.
I regret every moment that I spent working when my kids were that age. They are now 16 and almost 13, and they can still remember negative things that happened during that time period (not that they dwell on it because neither of them are "whiners" - these are just things that have come out in conversation). I never realized - at the time - how much they truly loved having me home. I didn't think it was that "important" since little kids don't require PhD's to "teach" them. But I have come to believe that kids really need a parent if possible - that's the age where so much internal development occurs.
My children are fine and well adjusted, but if I knew then what I know now, I would have never done it the way I did.
Children are resilient - remember that. I would guess that you are doing the best you can for your family. If there is no option to stay home then don't beat yourself up - just try to make the best of it (and make sure there isn't a latent issue with your nanny).
Good luck. You sound like a caring mommy.