Oh boy. First of all, know that I am incredibly biased when it comes to this issue (I am working on my Bachelor's in Alternative Health and hope to open a wellness center specifically for Women and Children). I have 5 sons, but knew nothing about organic foods until I was newly pregnant with my 4th. I was guided there by my aunt (whom we all used to make fun of for being so organic). Once I researched it all there was no going back. The short answer is YES, organic is better.
The problem becomes cost when your family is the size of mine (heehee). Since you have a small family right now, my personal opinion is that you really should go no other direction but organic. When I first got into it all, I had to shop at natural/organic shops, but now mainstream groceries and department stores are carrying organic formulas, baby food, bath and body prodcuts, and nontoxic cleaners. I'm so happy to see the trend. Be prepared,though, there will be people who think you're a nutcase (though there are less and less of those with every incident of food recalls). Just smile and know you are doing what's best for your child (and don't forget yourself as well).
PS. Do a google search for 'organic baby' or 'nontoxic baby' and that will get you started. These are good sites:
http://www.checblog.org/organic_food/
http://www.checnet.org/improve_main.asp
Best of luck!