You will probably have to get something custom made for that window. That is probably why the room is so cold...and probably warmer in the summer.
One thing that might work is keeping the baby's door open at night...that will help the warm air go in. Even it is just opened before the rest of the family goes to bed.
Another thing you can try is to have a warm mist humidifier in his room. My son has a stuffy nose so I was putting the warm mist humidifier in his room at night and at naptime...when I go in there to get him when he wakes up, his room is nice and cozy. If you do that, then keep the door shut so it keeps the warm mist in.
I have my son (6 months old) wearing those fuzzy footed PJ's that zip up AND a heavier sleep sack. He seems to be fine with those and the humidifier