R.J.
The cravings will go away IF the nutritional needs are being met elsewhere. You and your kids have wildly different nutritional needs (just as an example: your kids under 5 need so much fat in their diets that you would have a coronary eating the same things... they're putting that fat to use building myelin sheath to coat all of their nerves in their bodies). Adults need lowfat diets, kids need highfat diets... our mineral needs are different... sugars, proteins, you name it... put a kid on an adult diet and you get an obese kid (because they're always starving so they just keep eating and eating and eating trying to get the nutrients they need).
So before you start taking away where they're getting their fats/proteins/sugars/calcium from... make sure you have substitutes for ALL of the above. It's a PIA, but totally doable. Milk (dairy) is a super-food... so it's not easy to replace... but it's completely possible with a lot of nutritional research and work.