You seem to answer your own question and call it a "horrible habit" which makes me think you are doing this against your better judgment because it's easier. I would follow your instincts here and nip this bad habit in the bud.
I would definitely break this habit now. Otherwise, I see a future of bad restaurant experiences and eating in front of the TV. I've seen it happen with other people's children. As a person who had NO problems at restaurants with my children I can't tell you how amazed I am (and not in a good way) at how I see many young children behave at dinner time in public places.
No healthy child will starve themselves. If he's hungry, and you make it the rule that eating happens AT THE TABLE (high chair or not) and excuse him when he's finished-he will learn to eat a normal meal. This makes for more peaceful family meal times (not too mention less mess) and will prime you for having good public manners from your child sooner, rather than later.
If you keep allowing him to treat his meals as an optional and impulsive endeavor you are only making things harder for yourself later. But then again, maybe your standards aren't as strict as mine are for meal time behavior.
Some suggestions: 1. Follow a routine as much as possible-if he naps, plays and bathes at the same times each day, he's that much more likely to accept eating his meals at regular times. 2. As soon as you can (I realize he's young for this now) involve him in setting the table and serving his own food. Even something as simple as putting the napkins on can make a child feel more invested in the meal times.