Make a list for yourself. Then, each day, try to accomplish at least 2 items off of your list. (in addition to caring for the baby).
Then, feel free to veg a little. It is okay to be "lazy" sometimes. I happen to think it is a way that we deal with (or rather NOT deal with BY shutting down) all the little stuff that we can't get done. It's like when you take your newborn out in public and they "shut down" and go to sleep from sensory overload. You are overloaded and can't do it ALL so you are shutting down. :)
So make your list, and try to focus on only one or two items per day. You scratch them off when they are done, and you actually SEE that you DID accomplish something tangible. Balancing the checkbook is an invisible task...but it still must be done. If you write it as an item on your "list", then you can scratch it off and SEE TANGIBLY that you HAVE done something. Then you will feel less stressed and a little more deserving of some "down time". And you DO deserve some downtime. :)