I completely understand what you're going through. My son was not a good sleeper (and still not a great napper) either. Of course, I think if I had had more experience from the beginning, I might have had better luck. I'm not sure if you've read any sleep books, but "Healthy Sleep Habits, Happy Baby" really helped me. It really sounds like your son is just so overtired that being overtired is affecting his sleep. It's going to take awhile to get him beyond that state, but if you persist and be consistent, it will happen. It may take two days, or it may take two weeks, but please just try and be consistent either way. Come up with a routine that cues him that it's sleep time. Our nap time and bedtime routine are very similar, but we add a bath at night - even if he doesn't really need it, it just gives him a cue that bedtime is coming. After that, he gets all lotioned up and dressed for bed, we brush his teeth and read him a book or two, then we say goodnight to all his stuffed animals on his shelves (they lay down and go to sleep too), we sing him his favorite good night song while holding and swaying with him (rock-a-bye-baby - but we substitute "baby" with his name), give him hugs and kisses and put him in his crib with a soft, light blanket that he likes to hold. Also, we have a pull down curtain that my mother-in-law made that helps keep out the light. It really does help to make the room dark, especially at nap time! Sometimes he still cries, but not always, and usually only for 5-10 minutes.
At nine months, your baby shouldn't have to eat at all at night. He has learned that you will come to him when he wakes up and cries, and that's why he keeps doing it. It's something you'll have to unlearn yourself. Try one night to not go to him and see what happens. He might cry for an hour (or more) and then fall back asleep for 5 or 6! If you keep being persistent, pretty soon he won't cry at all at night unless something is wrong. I know he scratches himself when he's upset and that isn't fun, but part of the reason he does that is because he hasn't learned yet how to fall asleep on his own. Tugging on his ear might be his own comfort technique (or he may have an ear infection - have you spoken with his doctor?).
The other thing that really helped me was figuring out my son's best sleep pattern. Every baby is different, so figuring it out isn't easy. My son sleeps best when he goes to bed between 6:30 and 7:00 pm (earlier if he doesn't nap well during the day). I notice that the earlier I put him down, often the better and longer he'll sleep at night. This technique actually helps with the napping too, at least until he's beyond the overtired state. Now that my son is almost 13 months old, if he gets more than 12.5-13 hours of sleep at night, he sometimes won't take his morning nap and then ends up taking a shorter afternoon nap, but at 9 months he needed as much sleep as he could get in the day and at night.
Know that you're doing this for him just as much as for you! You will be a better, happier and healthier mommy when he's sleeping through the night. You can do it!