I'm with you. I don't "do" co-sleeping either, but I certainly don't begrudge anyone who does, but it doesn't work for us. I think since sometimes you arne't even being disturbed you need to just decide what you want and how to go about it. I am a light sleeper as well and am always disturbed by anyone or anything moving around in the night, in my bed or not! ;) It's funny b/c I wasn't really like this until I became a mom...maybe I'm just too paranoid! Haha!
My daughter also went through a phase when she would come in in the middle of the night. So, we had the talk when it was day time and light outside so she knew what to expect. I told her that if she came in we would take her back to her room and put her back in bed. I did let her snuggle for a few minutes, but then it was back to bed. At first she cried, but we stayed strong and kept doing it and it didn't really take that long for her to stop. Also, we taught her number recognition (she was just under 2) and told her that when the clock had a 7 at the beginning she could come in. She was actually really good about that and it hasn't been an issue since. Your daughter is 4 so she'll understand the parameters you set in place, she just might not like them! ;) But, stay consistent and caring and explain that you love her, but need your sleep (and some time to not worry about being interrupted if you're trying to get pregnant!). For me, that was the worst...constantly wondering if she was going to come in or not...I wouldn't sleep well always wondering and I never wanted to have sex for fear she'd pop in, so once we had a plan I felt a lot better about things!
Plus, I do think it's a good idea to think ahead like you said and consider how it will all play out if you have an infant in the future. It's hard to say b/c maybe she'd be over it by then, or maybe not. Ultimately, do what you feel comfortable doing. I know you have the added pressure of your husband, but if you don't want to do it, don't guilt yourself into doing it.