I personally think that starting solids at 5 months is crazy! No offense intended, I know a lot of moms do it. My boys didn't start solids until they were already walking (somewhere in the 8-9mo. range) and even then it was only experimenting for them. They've have a nibble of this, or a bite of whatever they could grab off of my plate. Both of my boys are fantastic and healthy eaters now. They even eat their brussels sprouts and they're only 8yo and 2yo! I think this is because I let them tell me when THEY were ready. They really don't NEED solids for the first year, so why force them?
Some safety guidelines though, if you are going to feed solids early.
Your baby has to be able to hold his head up independently, this is very important. If he can't then he's at great risk of choking.
Really, he should be sitting up independently before solids are started.
Always supervise him closely. I let my boys drink from a cup (about 9-10 months) before I let them handle their own spoons (vice finger foods)
Pay close attention to his stools. Baby's digestive system is still immature at this age and anything with harsh proteins in it can cause intestinal bleeding. (another great reason for waiting to introduce solids!)
And yes, you shouldn't be putting the rice cereal in his bottle.
Let him experiment and try all kinds of different things. Mashed bananas are a popular first food. Don't stress about giving him a balanced diet this early, plenty of time for that later, after he turns 1.
Hope I was able to at least give you a different perspective! :)
_________________________________________________________
And another thing. I think you should find a new pediatrician. Yours doesn't sound very experienced. Any pediatrician worth his/her salt knows that you can't put an arbitrary age on developmental milestone. You can give a range, yes, but every child is different and new experiences for baby have to be weighed against how far baby has come on their own developmental scale. For your doctor to say it's ok to start at 4 months (arbitrary age line), well, that sounds negligent and lazy to me. I'd look for a new doc ASAP.