My 1st question is, Why is she wanting to pull him out of regular school? Is it behavior problems, bad grades, personality clashes? Or is it that he's bright beyond what they are able to teach him at his school? Either way, it is best to do ALL of your homework before she just pulls him out of school.
Not to bash a previous post about A Beka, but my mom worked in the school system for 20 years and dealt a lot w/home-schooling. She just told me the other day (again) that if I ever heard of a place offering that curriculum to go the other direction. We have a cousin who pulled her kids out of public school and they were thrilled that the kids were making straight A's -- the only problem was that the curriculum was about 2-3 years behind what they should have been! YIKES!!
I suggest your sister research as many curricula as possible, as in-depth as possible. A great help would be to check with the school system where she lives. The counselors there should have a wealth of knowledge on the best home-schooling programs. They could even refer her to other home-schoolers and their parents who can share info as well.
Your nephew CAN go to college after home-schooling, and I suggest that y'all push him to do so no matter what ends up happening! HOWEVER: Home-schooled kids have to make higher on entrance exams than public/private schoolers. For example, my alma mater (in Mississippi) has a minimum ACT of *16*. . . unless you were home-schooled. Then they have a minimum requirement of *20*. (I checked with one of their counselors for one of my home-schooled cousins that I mentioned.) That 4 points may not mean anything to a person who tests well and has a great background
It's not too late to change to home-schooling, and it can be a great thing and offer lots of opportunities that you can't always get in a "regular" school. But there is a LOT to know about it, and several things that can trip you up if you want to end up working anywhere outside of a KFC cook line.