Anxiety is not logical, so you can't talk yourself out of it. You also may not be able to address it without help. Ask your primary care physician for a referral to a counselor who accepts your medical insurance. This is no different from a referral to a medical specialist - very routine and very simple to accomplish.
It might help to write some notes though, to share with the professional. Were you always nervous when they were little? Did something happen after they were adults that made you experience loss or the fear of loss? Do you feel a loss of control now that the kids are adults and you aren't in charge or watching over them every second? Are there other things that make you anxious? You don't have to figure all this out - but writing some things down can help you organize your thoughts about what you might want to discuss or share.
And don't worry right now about who doesn't want to be around you or what effect this has on others. Get help, and focus on yourself. This is treatable, and you will be able to turn this around. You will. But you need objective help to make that happen.