It sounds like there is no real schedule between you and your babysitter. That's the first thing that he needs. Every morning he should have to get up at the same time. Breakfast, play. Snack, play. Cleanup. Lunch. One hour in his room having "quiet time", by himself, with the door closed, no exceptions. Play quiet music. He can read, play with toys, or lay down and sleep if he wants to - no pressure from you. However, he cannot come out of the room. If he's making a bunch of noise, don't take the bait. (A nanny cam would be smart to see if he is actually in trouble - if you've got a safe room for him, I would think he'd be fine...)
Getting him up in exactly one hour. Snack, play. Dinnertime, time with daddy, bath, short bedtime routine, bed. Bed AT THE SAME TIME everynight. No allowing him to come out of his room either.
As you tighten up his schedule, his sleeping habits will get better. It might get WORSE before it gets better, mind you, but you can't let him fall asleep on the floor any old time during the day because it throws everything off.
You may not be able to really institute daily napping after a year, but the quiet time every day may lead to him snoozing after he accepts that he is in his room without fail, whether he is with you or his babysitter. What you have to do is get your babysitter and you on board with exactly what it is to be done and expect it of yourself and of her.
And NO, don't drive him around to get him to sleep. Being a slave to your child in this way is not the way to set up healthy habits of any kind and a sure fire way to switch the dynamics of parent over the child to child over the parent. Believe me - you don't want to go down that slippery slope because it's a short one...
Dawn