Hi Vicky!
Dont stress too much over it, since this has nothing to do with you or your mothering style, but rather with the needs of your son. He needs to be with daddy as much as he does with you, he just gets him in small dosage so he has make the best out of the time he has with him. You said that dad travels, well, that's it,your son needs that extra time and dedication because he knows he'll be gone at one point.
My kids did and still do the same, since my husband also travels. You said he is very attached to you when he is not around, so see? "is not you". Like one of the other moms said, he feels free and confortable enough to express his feeling and do what his little heart desires, dont take it personally, he (your son) is not doing it tu hurt your feelings, he just needs time with his daddy.
The other day I was at the supermarket with my 4 yold twins and dad met us there to say good bye before his business trip and as soon as they heard dad was going to work they said- No, Mom YOU go to work! dad stays! bye MOM!!- yeah...they were insisting in me going instead if him. Now I've always felt so bad for them when my husband has to go, beacuse I though they would be used to it by now, but no, they never have gotten used to it and my oldest is 9 years old..So when dad is back and they climb all over him and want to just be with him I give them the space and use it for me ( I promisse you I can really use the brake!)
I understand how you feeling though, but beleive me, is very normal, I'll be worry if he didn't want to be with daddy, but only with you when he is around (dad), because THAT would show that he is becaming too dependant on you and that's not good. They are bonding during that time, rejoice in the fact that those moments are the foundation of their relashionship for years to came.
I hope this helps!
A.