I find it hard to even find something to judge you on!! Don't worry so much about being judged. Honestly, it sounds like you're really going through a trial with him, and that you've done everything a compassionate mother would do!
Beyond taking him to a doctor and discussing this, I just have an observation.
Some children don't care for sleep. I didn't. I would lay awake in my bed until 12 o'clock most nights at this age and on. My mother was NOT compassionate about it...so I went to bed at 7 or 8 or whenever, and just laid in the dark with no stimulation for 3-5 hours. I would get soooo bored I would count just to pass time, and as I got older, I exercised instead. This honestly went on until I was in high school, when I just locked the door, turned on my music low (but she still insisted she could hear it) and read until 12.
Both of my children are this way. Doesn't matter what time I get them up at, they both have a hard time falling asleep. I put my 6 (soon to be 7) year old to bed at 8, and she reads with a low lamp on until she falls asleep...usually by 10. I wish it were earlier, but it's not. My 20 month old doesn't fall asleep until after her older sister, generally.
I am grateful I homeschool, because I think my six year old would be a very different child if I had to get her up at six every morning. And believe me, I gave that a go...got her up that early for several months in a row, and she NEVER adjusted to going to sleep any earlier.
I'm sorry I don't have any advice, beyond checking with your doctor...I just honestly think some people don't go to bed well. I still don't, and I'm in my 30s. I get up at 7 every morning, but I still won't go to bed until after 1 am. I'm not really that tired most days, either...some people seem to need less sleep.
Oh! I forgot. As far as the manipulation goes...I guess I'd try letting him have soft music at night (we rotate...classical, Christian, reggae, Christmas, the Archies, Smokey Robinson, the WallE soundtrack, etc...) and letting him read all the books he wants to. THat's my rule...stay in bed, but have all the music and books you want. :)