First, let me suggest that you change your languge just a bit if you do not want to offend anyone. You do not mean "normal" you mean "typical." Disabled people are normal. How we treat them is not always so, but you have a chance to make a difference here, but not for the kid you think I mean.
I am not going to make many assumption about what the nature of Bob's disablity, but most people with developmental issues are much younger in thier interests than they are in age. It is very possilbe that his maturity level is quite close to your son's. With the appropriate supervision, there is nothing wrong with a freind relationship continuing between Bob and your son, however, you need to do some leg work, which I highly suggest you do before it continues. It would be a shame to end a friendship that two "kids" enjoy based on your missunderstanding.
If Bob is disabled, he should not be out running the neighborhood anyway (another story) so I would ask Bob to take you to introduce you to his parent(s) and tell him that you are concerned about his safety and you want to meet his family before he plays with your son anymore.
Explain the situation to the parent(s), tell them that you are concerned that Bob may need a little more guidence and supervission around the neighborhood and ask them directly if he has any issues that would be a bad influence on your son. You need to know if he has any sexual behaviors or if he has any issues that have needed more than typical intervention. Ask them if they would be willing to communicate with you when ever Bob would like to come over to play so that one of your can be there to supervise, and tell them how often you can have him over if you still feel like you need to limit his visits. If you aren not satisfied with the anwsers you get, then, by all means, end the relationship, but that would be a terrible loss for your son. Thats right, for YOUR son, and Bob, but mostly, for your son.
Kids who are involved with children who have neurobiological developmental disablities develop respect for them. They learn that children with disabilites are more like them than they are unlike them. Kids with disabled youth in their classes score higher on standardized assessments than kids who are schooled with only "typical" kids. They learn to tolerate minor differenences, and they learn terrific leadership and stewardship skills about how to view and care for thier fellow human beings. They learn that "normal" takes many different forms and that is priceless for any typical kid.
I can tell you a few things about children with these kinds of difficulties. He does not have a clue that he did anything inapproriately by walking into your house, and you should not hold it against him that he did so. He may need to be taught each and every social expectation in every new environment because he cannot generalize "the rules" in a new place like you can. That is part of many developmentally disabled issues. He probably does not mind if you explain it to him either. In this way, he is not typical, but his desire to interact with other human beings is probably as typical as yours or mine. I can promise you that he has been rejected before, and that would tend to explain why he is overly invovled in your household and is a testament to how nice you have been to him so far, I would not yank that rug until I was sure there was something to be afraid of, chances are good that there are not. Tell him how often it is approrpriate for him to come over, and don't be surprised if you need to remind him, it may be a little anoying, but he really does not mind and you probably cannot hurt his feelings by just telling the truth. It is likely he has not heard the truth very often, and they need it more than typicals do because they cannot figure that out by themselves.
Finally, treat him like you would anyone else and expect good behavior from him, tell him what he does that is not appropriate. Share this with your wife, and tell her to get over being scarred, there is probably nothing to be afraid of, unless you get answers you don't like from the parents. You have an opportunity to be a great example for your son by giving him the gift of tolerance and kindness in how he sees you treat Bob.
Your ball...
M.