L.D.
I wonder if something happened that has caused him to feel unsafe while you are not around. If I were you, I would probably delve into this area of concern a little bit more. I'd just spend more time with him, casually asking him questions about what is going on with each of his friends and seeing if you can flesh out any information that would indicate that maybe somebody had been unkind or even bullied him at some point in time. I would urge you to try to appear to be as casual and conversational as possible so he will feel comfortable sharing this information with you.