I heard from a local farmer that locally grown foods have fewer pesticides and preservative-type chemicals, because they don't have to be shipped. Not sure if this is true, but I DO like to support the local folks.
As far as organic goes, Parenting mag does an article or 2 every year about which fruits and veggies to focus on. Check out http://www.parenting.com/article/Mom/Health/Eating-Green-...
In a nutshell: buy organic peaches, apples, strawberries, potatoes, soy, milk, eggs, and ... ketchup! (because kids eat so much of it) I also do organic carrots and spinach, due to high nitrate? levels (it's been so long I can't remember!).
We don't count the cost difference for the short list above. We DO buy organic when we see the cost difference is negligible, but we go for the regular stuff when the organic is at least 50% more. Watch your brands too! For example, where I shop, regular Kraft mac-n-cheese is about $.75-$.85. Annie's organic is $1.10. Kraft organic is $1.50! We only buy it once a month, but we buy Annie's every time. Of course, fresh veggie prices fluctuate constantly, so it's hard to say what's more expensive. I do know that carrots are roughly the same price either way.
And here's a tip to keep down costs: Super Walmart. I know, it's the evil empire, but they have great prices on lots of stuff, including organic produce and dairy (and soy milk!). We still shop at "regular" grocery stores for certain things (like my daughter's SUGAR FREE Cascadian Farms O's), but over a month, we spend about $500-$600 (food, cleaning products, toilettries) for our family of 3. We eat the full 5 servings of fruits and veggies every day, exclusively whole grains, and not much processed food (except granola and ice cream). It's going up, of course, but I think we're doing a good job with our budget. Good luck with yours!