Hi, S.. Well, I have a few suggestions that may help. First, I think that for a child who is reluctant to go to sleep, it is important to get that child tired enough to want to do it. Try making sure he has enough physical activities later in the day so that his body will need to rest. Also, make sure that he doesn't eat or drink anything very sugary at or after dinner; that includes sodas, which also have caffeine, which of course will give him tremendous energy and keep him alert at the wrong times. It might help to model the fact that this is bedtime for everyone -- if he sees that you are putting him to bed but you are staying up to do fun things, he will feel left out and want to join you. So maybe you should put on your pajamas, too, and make sure he knows you're going to bed, too, and that everyone needs to do this right at that time.
I would also start limiting his nap time during the day. Don't take it away totally just yet because if he gets overtired, he will be cranky, upset, and unable to sleep because he is overtired. But I would start waking him up 15 minutes earlier than usual for the first week, then gradually a little earlier than that, so that he will become tired toward the end of the day because he did not get a huge block of sleep during the day.
I don't understand what you mean about taking the gate down off his door. You mean the baby gate? Maybe you should put it back up so that he can't get out of his room when he's naughty at bedtime. If he manages to sleep on the floor, that's not a horrible thing as long as the floor is clean; eventually, he will get the message that his bed is more comfortable than the floor, and he will prefer the bed. I know that sounds harsh, but it's not because you are not punishing him or anything. You can always pick him up and put him in the bed if he falls asleep on the floor.
Another thing -- it sounds like he's getting a whole lot of positive attention for disobeying his bed time! Make sure you don't play with him or give him much attention when he does this behavior. Holding him and singing to him, which don't seem to put him to sleep at all, are giving him a reward for staying up past your limits, so he has a really good incentive for breaking your rules. Make sure he doesn't get a reward in any way for disobeying you.
Also, unfortunately, a lot of children do not take mothers seriously as the disciplining parent. It might be necessary for his dad or some other authority figure to put him to bed and lay down the rules. There is no harm in this. Maybe if both of you do it together, he will understand that you both are united and you both mean business. This may especially be true if he is used to getting his way by running from you to Dad, who does the opposite of what you want your son to do.
All kinds of things can be factors. There might be some kind of noise in the house or coming from outside which might be energizing him, too. You might be able to minimize that by keeping the TV off at night until he falls asleep, or by soundproofing his room somehow. On the other hand, he might be agitated because it's too quiet! In that case, you might even get him a little radio with headphones or music player to keep him quietly occupied until he falls asleep. I know there are things that go into a child's crib with soft lights and music which soothe a child to sleep; this kind of thing can be mounted on the wall, if necessary, to help out.
Well, I hope these ideas are helpful to you, and that you all find a way to sleep peacefully very, very soon!
Peace,
Syl