There is always one in the bunch who is a power house and knows it.
You seem like you are doing all the right things, but for the wrong personality. These tactics are perfect for children like your oldest. But other tactics work for his personality.
You need to develop a good gift to ignore. Literally! This will only work if he is not getting into harmful things. If he is doing that as well, ignoring him sometimes and not when he does those things, will result in him only doing those things to get your attention. So be aware of that.
When he screams and cries, ask him what he needs. If he is crying just to cry. Tell him to take sometime to work it out, and walk away. When he calms, ask him if he is feeling better. When he stops engage. He will learn to use words, they get him more.
When he is disrespectful, to anyone. Everyone should learn to say, " I do not like it when you speak to me that way, or do that to me. I will not speak to you when you are like this. " And ignore him. He will learn that his behavior will cause him to loose interaction. And when he is nice, people like to be around him more.
Dinner. There is a set time for dinner. He does not get to choose. So when he takes two bites, and says he is full you ask him to take one more good bite. If he refuses, let him know that there is no snack after dinner, and that if he is not really full, he should finish with the family. If he refuses. Let him up, clear his plate. When he says he is starving and needs a snack, remind him of dinner. He is not starving, you know that. And what he is doing is eating enough to let him off the hook to sit and eat something, that is not going to be as good as the gram crackers he will get later when he claims he is dying. You feed him three times a day plus snacks, he eats well. Trust me, he will do this for three to four consecutive dinners, and then he will learn that it is dinner, or rumble tummy. (Kids who don't eat dinner NEVER get dessert.) Make his favorite on the nights he chooses two bites for dinner.
He is stubborn. So you will be dealing with this until it breaks and he sees that he will not win. But he will replace dinner battles with another one, because he will decide that there isn't a great pay off to not finishing his meal. So you keep at it. Do not stop for anything. If it takes a month to break bad behavior, you can not stop. If you give him time out, and he does it again tomorrow, give him time out tomorrow, and the next day. You are not kidding, you do not tolerate this behavior.
There is no set amount of time that discipline works. Sometimes it is much longer for children to realize hey, this isn't worth it. But if you stop because you have been doing it for a week and nothing has changed you have taught him, that patience is a virtue. And kids with his personality are realllllly patient. I know you have seen that. So stick to your guns, and don't give him his way unless he earns it.
Good luck.