My husband and I have basically done the same thing as Tiffany. We give our daughter a wide variety of foods, and let her try things even if we aren't terribly fond of them or if they are unusual and aren't sure if she will like them. We also did not let her have anything sugary (no cookies, candy, ice cream, dessert) until she was one. She got a little bit of cake and ice cream at her party. Occasionally I will give her a small bite of something sweet, but she doesn't get cookies or anything else loaded with sugar. I want her to be healthy and learn to like good food, not the junk food. I am hoping this keeps her from developing a sweet tooth, and helps her enjoy many different healthy foods. She usually eats everything that we eat- whole wheat bread, sweet potatoes (without the marshmallows), broccoli, chicken, almost everything. Definitely give her a wide variety of good foods so you aren't going to get into that "my kid only eats...(3 things)" slump when she is a toddler.
As far as the expense, it is a pay me now or pay me later type of thing. Either you buy and eat healthy food now and stay healthy, or you buy junk food/unhealthy food now and pay for later with medical bills (medications for high blood pressure or high cholesterol, doctor's bills and hospitalization for a heart attack or stroke). You can certainly eat healthy on a budget. The key is buying the right amount of fresh fruits and vegetables- so they aren't rotting in the fridge before you use them (yup, I'm guilty of that at times). Also, consider getting a membership to Sam's Club or Costco. The meat and fresh fruit/vegetables alone are worth the cost because it is so much cheaper than the regular grocery stores. Oh, and they have cheap diapers, wipes, and formula- good stuff, but the most inexpensive I've found (Member's Mark brand)-definitely worth a membership for that!
Another recent discovery for me is the George Foreman grill. I got one for about $30-40, and we use it all the time now. It cooks hamburgers, chicken, steak, beef, fish, whatever, and it does it very quickly, easily, and healthily. Just sprinkle some seasoning (I like Grill Mates Montreal Steak and Montreal Chicken seasoning- in with the spices), pop it in, and in a few minutes it is done!
Just start substituting things here and there, and serving healthy meals to your husband. He'll get used to it, and probably thank you for it later. If you eat healthy most of the time you won't feel as guilty when you indulge in an old favorite.
Good luck, and happy eating!