I just wanted to say that my son has spent most of his life with just adults. Sometimes the grandmas are around, sometimes both parents, but since we have no friends or family close, my son is usually alone with me, and with his father when I take a break. I cannot be engaging with him every moment we are together. I felt a lot like you, thinking that he needed to be around more kids. He's now 3.5 and he is doing fine interacting with kids his own age. When he was younger, he preferred adults. Mostly because he was very social early on and never wanted to parallel play. He has always wanted to interact, but kids his own age preferred not to.
I think it's normal to feel like you do. I don't think stay-at-home moms would have started playgroups if they didn't want the interaction. I joke that playgroup is more for me than for him, but in a way it's true. We go through phases. For a while it seemed like all I did was read books to him. Then we went through a puzzle-building phase. Then he was really into his laptop. Now he really likes his dvd's. Half the time he isn't even watching them. He just likes to stack them like blocks, or lay them out side-by-side like some sort of puzzle. Now he is really into a pretend phase where he pretends to be one of the characters he has seen on his dvds or in his books. He thinks he's a car and is always revving his engine. Crack us up. He helps me with the laundry and the dishes. I give him little things to do along side me. One day a week our "activity" for the day is picking up all of his toys he has left around. That's play to him.
When I need quiet time I tell him so. When we get up in the morning he is only allowed to deal with things that I deem low key so that I can start my day quietly. When I want him to settle down for bed, we dim the light to wind down and he again is only allowed to do quiet things.
Not sure if any of that helps. But, I'm sure you will find your own rhythm. You can't be on all the time, and it's okay to set boundaries so you can maintain your sanity.
P:)