OK, this might sound silly. I do not have children. I read this site, and others on infant care, because I work in animal rescue, and I often have young kittens (and their mothers) in my care - very similar to caring for a newborn baby. Mothering is mothering, whether baby has 10 pink toes, or 4 furry paws and a tail.
Anyhow, I always side with nature unless there's an obvious problem. Vets (and doctors) tell us that kittens (and newborns) are "supposed" to nurse every 2-4 hours 'round the clock. Well, some do, and some don't. They all seem to have their own schedules. I have a mother kitty with 5 wee kittens right now. 3 of them nurse all the time, at least every couple of hours, just like the vets say they should.
But, there are 2 who don't. They scare me by refusing to nurse for 12 hours. Just about the time I panic and get ready to break out the emergency bottle and special kitten formula, they decide to latch on & suck away for an hour.
No worries. They're gaining weight and developing normally, and their bowels are working fine.
I think Katie's advice is good. If you want baby to eat more often, wake him/her up to eat, and watch the bowel movements. If it's going in and coming out, all is probably well, even if the schedule isn't exactly what the doctor ordered. If you're really worried, call your doctor and ask, for sure.
And be happy you aren't a mother cat! They don't change diapers - they eat their babies' waste and pass it with their own. Yuck!