First of all, both of my inlaws suffered extremely abusive, neglectful and similar childhoods as your mother. They are both incredibly loving, bright and intelligent people as a result. Your mother has chosen to be unhappy as her form of coping and is having a hard time breaking out of that cycle. You can't keep using that as an excuse for her nasty behavior, and neither can she.
Secondly, you sound as if you almost blame yourself and are trying to win her over. Stop that, you don't need to feel guilty or like you are some thorn in her side. You aren't, even if she treats you and your children like that.
Third, if you want to help her, she could possibly benefit from professional counseling. You can lovingly sit down and tell her that you feel she could find some closure and healing with a therapist that can work with her. I had to do this with my own mother who went crazy and suicidal when my father died of cancer, so I know it is tough. I also know that a close relationship with Christ can offer healing as well. After that, stop pestering her. Don't make allowances for her rudeness, or allow it.
Fourth, take care of you and your family. Your husband mistreats you and your kids. Work on that. Leave him if you need to or get marital counseling together. This, I think, would make her (and most likely you and your children) very happy as well.
I recommend you read the book, "How to Hug a Porcupine: Dealing with Toxic and Difficult to Love Personalities"
http://www.amazon.com/How-Hug-Porcupine-Difficult-Persona...
A few very short videos (only a few minutes long a piece) on healing through Christ that you can both watch together. Each one is unique and beautiful and may help prick something in her heart:
Lifting Burdens:
http://lds.org/ldsorg/v/index.jsp?autoplay=true&index...
Hope Ya Know, We Had a Hard Time:
http://lds.org/ldsorg/v/index.jsp?autoplay=true&index...
You're Never Alone:
http://lds.org/ldsorg/v/index.jsp?autoplay=true&index...
Infinite Power of Hope:
http://lds.org/ldsorg/v/index.jsp?autoplay=true&index...
My Burden was Made Light:
http://lds.org/ldsorg/v/index.jsp?autoplay=true&index...
Good Things to Come:
http://lds.org/ldsorg/v/index.jsp?autoplay=true&index...