When my daughter was 4, my father-in-law passed away. We knew he was sick, but he looked and felt fine - then he had a day he didn't feel so well, then he passed away. We were all reeling and scrambling and during all that mess, we had to deal with my daughter's questions. She still asks some - it's been 2 years.
We were honest with her. We told her that Grandpa had been sick. How do you explain cancer to a 4 year old - I don't understand it myself. We told her that there was a bad thing that grew in Grandpa's tummy and that the doctors couldn't take it out, because it was to big.
We told her that Grandpa was in heaven with Jesus and that Jesus made Grandpa feel better - he doesn't hurt or feel bad anymore. Then we told her that Grandpa loved her and he will be watching her from heaven. We have a picture of my husband's family that was taken less than a month before my father-in-law passed. We keep it out where it can be seen. I told her "I know that you're sad that Grandpa is with Jesus, but if you want, you can look at this picture and remember fun things about Grandpa and you can tell this picture how much you love Grandpa and miss him".
It won't be easy. And there will be some questions asked over and over again. Just be there. Tell the truth - I don't mean go into total medical term mode with the whole way doctors talk. Break it down to a level your son can understand. Don't hide things that belong to Grandma and don't put pictures of her away. That makes things seem bad to the kids.
I wish you the best. This is a terrible situation to be in. All those parenting articles NEVER have information on this kind of a situation. You kind of have to wing it.