First of all, try thinking about things differently.
A). Your ex said he won't fight you for wanting to move. Stop thinking that means he doesn't care about your son and check that problem off your list.
Do you want him to fight you about it? Would that make you feel better? My daughter's father had no interest in her, didn't pay support, BUT, he and his parents tried to stop me from getting married, they tried to stop me from being able to move when my husband got transferred with his job. They fought me every step of the way, in court, for anything I did to get on with my life and make a life for my daughter. That wasn't a "signal" that he cared for her. P.S.....She's 23 now and he STILL isn't a father to her.
If your ex isn't going to fight you, be glad about it. Get it in writing. Have an agreement that you will be flexible, as will he, about parenting.
B). Your mother has little to do with it unless she is the one who is supporting you right now and she has fears that this is an impulsive decision on your part and you are taking your baby somewhere that you won't have a support system or know where you'll live or work. If that's the case, as a mother and grandmother, I'd be worried too. But if her only concern is regarding the father, that has nothing to do with her.
It's between you and the father.
You can't make a father be a "dad" no matter how good a mother you are.
Your son is only a year old. There is every chance that when your ex gets his life figured out, he will want to be there more for your child.
At that point, hopefully you can let him, without a fight, just like he's not giving you a fight right now.
Have your ducks in a row...know where you're going and how you'll get there and have a solid agreement with your ex so he can't come back and say you took the child without his permission.
Don't worry about what your mom thinks and don't worry right now about when your son is a teenager. Trust me, a lot will happen between now and then so concentrate on now.
Best wishes.