Its totally realistic to feel guilty when you quit before you or baby are ready. Its part of the bonding experience that you're both going to miss out on now. However this is for your health, you're not being selfish, so you're making the right choice to stop and go back to your medication.
My dear friend is bipolar and struggled with sanity during the time (a few weeks or months, i can't remember how long now) she went off her medication to breastfeed. She felt so guilty going back on her medication, but now she's the mom she wanted to be. Breastfeeding is wonderful, but it is not EVERYTHING.
I had to quit early with my first child. She was born using a vacuum extractor and apparently that affected her cranial nerves that control sucking. We lived in KY at the time and in that area hardly ANYONE breast feeds, so the nurses at the hospital do not know how to help you and there is a lactation nurse at the health department, but she serves 5 counties, so you get her once a week... Anyway, my child did not latch on for 4 days, and even then it was incorrect. We had every breastfeeding problem in the book and at 6 weeks we discovered (after 24 hours with a dry diaper) that she'd lost 10 oz more then her birth weight. CRISIS! The pediatrician wanting to avoid a hospital stay would not allow me to breastfeed until she'd had x oz. of formula first, and my husband refused to allow me to use anything other then a bottle. Even then, it took over 2 hours for her to drink 3 oz. and to try breastfeeding at that time when she was exhausted and full, just was unreasonable. For 2 weeks I spent every waking minuet feeding her or pumping, and I didn't get much. Then she began to reject the breast and we were basically done. I was devastated. I spent a lot of time blaming my husband and myself. But honestly I had to get to the point of being thankful for her life, that she was not a failure to thrive baby, and realizing that formula was made for these types of situations.
The best way to deal with the guilt is to talk about it. Very much like you've done here. The why of your guilt is that you're a mom, you're designed to want to nurse your baby longer, you've got extra hormones in your system making your more attentive/sensitive to your baby and his needs. Logic is not primary right now. You can look back and see that its the right choice, you can understand it right now, but you can't FEEL it. Keep talking, keep going through your reasoning. Identify what feelings are true and which ones are lies. Speak the truth to yourself and discount the lies. And choose carefully who you talk to. Seek people who understand and are supportive.
It should get better as time passes and your hormonal levels go back to normal, also getting back on your medication should help too. In my case it was a week or two. I'd been dealing (without knowing until it happened again with my second child, and no BF issues) with PPD in the form of anxiety, and once I stopped BF all of that went away. The BF issues triggered it. With the second baby it was brought on my nothing, just time of day, and it ended in 2.5 weeks after the baby was born, and we breastfed for 14 months.
Best wishes!