The Red Cross has a Babysitters Course that is for 12 year olds so I believe a 12 year old is allowed to babysit. Perhaps you can start with just trips to the store or something similar to help teach the step son to be responsible enough to watch the younger siblings. It may take a few months to see the results you want but with short trips you can at least start.
I also agree you need an adult to watch a teenager. Even if they are not responsible for watching him it will come to a point when they will tell him to do something, like turn down the TV so the little ones can sleep, to pick up a book, cd, or something he left in the way, and it will be an argument in the making because he'll say "You can't make me do anything, you're not my boss" and the teen babysitter will say "I am trying to take care of the little kids and you are making that hard"...etc.
In my opinion he is old enough to babysit and it's up to his parents to teach him how to do it and be responsible.