I understand the frustration building into anger because you have other things to get to or you're tired, I've experienced that myself - still do on occasion. You have a good routine but it sounds like the baby is changing it for you. Maybe he isn't as tired and needs his nap later.
When you see that it's going to be one of those days, you could try taking your older son to his room for quiet time before you start struggling with the baby (if he's still awake), at least that way he'd be out of the line of fire when you start to get frustrated. And you wouldn't have to explain what's going on, why the baby is crying, etc.
I used to intentionally do things with my girls to tire them out in the mornings so they would nap, neither were good nappers if not manipulated. Swimming is a good one in the summer (if you have a pool) even for babies, lots of walking, big lunch, my youngest liked to do marine style pullups at 9-10 mos so I would encourage her and count them (she could do 20+). If he's still taking multiple naps watch the duration of the morning one, and see if that is causing the afternoon problem. Some people don't like it but you could work on encouraging the younger son to have quiet time as well by putting some toys in his crib and letting him play for a while and then coming back to put him down again a little later. Or setting up a blanket wherever you are and letting him play quietly there.
Falling asleep myself was the trump card, I often would lay her next to me in my bed and let her nurse. I would pretend I was asleep and since there was nothing else to do my daughter would follow suit, then I'd leave her where she was for her nap - they eventually 'pop off'. Sometimes I would fall asleep too, rarely though I was always thinking about what I needed to get done.