Wear your baby! Especially when they're this small, babies are SO easy to wear in a sling or wrap. Plus, you don't even need to bundle them up because you can zip him up in your coat. This is how we carried my son all last winter (he was 5-10 months old).
I realize that I have maybe a larger collection of baby carriers than many people but here are some of my favorites:
1. Kangaroo Korner adjustable fleece pouch. This is a GREAT carrier for babies up to about 20-25 lbs. They can face in or out. It's warm and cozy. Wear it under your coat, put a hat on your baby, and zip him inside. Great for grocery shopping etc. Slips on and off very quickly so it's good if you're going from a car seat into a store or whatever.
2. Storchenwiege Wrap Sling. This is the most comfortable baby carrier for babies up to 40 lbs (although now that my son is a toddler he wants to get in and out too often and this one needs to be tied, so we don't use it too much). Great for longer walks, or if you're taking the bus/train and want to just know that he's snuggly and safe. We took my son to Germany/Austria when he was 2 months old and hiked about 40 miles with him in one of these carriers. We did not take a stroller on the 2-week trip. Love this carrier.
3. Ergo baby carrier - this one is great for older babies (like 6+ months) - it's what I use most of the time now, but it doesn't go under a coat as well, and it's not great for very small babies.
So... I'd ditch the stroller and wear the baby. Put together a backpack as a diaper bag, and get on with your life! Congrats on your new boy :)