My son had Torticollis as an infant. So his head turned naturally to one side. So when he lay in his crib (or anywhere) or even just in his car seat, his head always looked one way. He developed a flat spot on his head.
We took him to physiotherapy and I had to do exercises with him every day to strengthen and relax his neck muscles - so that eventually he would look straight on.
You do want to do something about it, because it can lead to a permanent flat spot.
Some people recommend wearing helmets. I could ours in time (around your baby's age) and we were ok. He still has a flat spot on his head, but unless he goes bald at some point, it's covered.
I would definitely talk to your pediatrician about it.
There are devices you can buy to put in the crib (that are safe) to encourage them to change head position.
The big trick I did (along with the stretching his neck to go the other way) was to always situate him so he had to look the way he didn't want to. It felt a bit cruel, but it helped strengthen his neck.
I don't know what is causing it in your case (may not have anything to do with neck) but tummy time, also bouncy seats (again move him around so not always looking one way), etc. can help.
When you put in crib, you can also change it up - so head one direction one time, next time put his feet that way.
Good luck :)