Well you've made it this long... You love the house, in 6 short years, your children will begin to transition out of the house which will lend more room to those left at home.
I would think, financially, it is easier for you now to not have a mortgage and just be responsible for property taxes and routine upkeep on the home.
It sounds like you have a home with a lot of character!
If you think about it.... YEARS ago, as I was growing up, it was completely normal for a family of 4+ in a home that was 1000 SF or so and share a bathroom.
Think about when all of the children are gone.
Example.. we have a very nice custom home and our daughter had the entire upstairs to herself while growing up. She is 20 now and lives in her own 2 bedroom condo which is about the size of my upstairs... 1500SF . So what do I have now?? An empty 1500 SF upstairs. Hubby moved his office up there and I might go upstairs once a week. If we have guests, they have the area to themselves.
We've considered downsizing but we would never find a lot like we have now. We have a rare lot, especially for our area which gives us privacy from neighbors. MOST of the homes around here are zero lot lines or so close that you are on top of your neighbors all the time. I hate that.. if I want to I can go out back naked and no one knows because we back up to a huge nature area that will never be utilized for homes or anything other than nature preserve.
We hate no privacy so that is a major factor for us staying here plus, I love the wildlife and if we were to downsize, we'd end up paying more than we do now which is nothing for something 1/2 or less the size. We keep a small mortgage for investment and tax reasons but more than likely, our house will be gifted to our daughter because when she is at a point to purchase a house, it will be very difficult. Our condo is in a prime area and already gone up significantly since we bought it and we have no mortgage on it. However, the traffic in that area is brutal. It does work well for our daughter for now though and it will end up being an investment property if she ever decides to move out... she LOVES it.
There are many factors to consider and I believe if you love where you are, you find a way to make it work.
As for the glass doors, go install solid wood doors (probably less than $1000 installed) and bam... private room!
Best wishes to you. I know it is hard to make decisions like that. We love our home and do not plan to downsize and settle for less than what we love.