I would wait until the time comes. I don't think a 3 year old can understand without an example and still their understanding is a bit sketchy. My grandfather died when my son was 3. He had had alzheimer's for a number of years and we knew he was not going to be with us much longer. My family visited him frequently even though he did not know who we were or what was going on around him in general. My children would make him pictures and tell him about what they were doing. Like I said, I don't know if my grandfather even knew what was going on around him, but it was what I wanted to do. Well, after he died, we took my son to the viewing, but not the funeral. We talked to him on the way to the service telling him that grandpa was super old and when you get to be super old your body/brain doesn't work anymore and you die. We explained to him that we were going to a service to honor him and share memories of him and then he would be buried in the ground and we would never see him again. We tried to be as straight forward as we could. He handled it amazingly. It has been two years now and every once in a while my son will ask to be reminded what happend to grandpa. We drove by a cemetary the other day and an elderly man was sitting out next to a fresh grave. My son asked if he was talking to our grandpa. It made me smile because I think he finally understands. He knows we will never see him again, but we can visit his grave if we want.
Sorry for the long story, but after going through this experience I truly believe the best thing to do at that age is to wait until it happens. Children only understand what is happening right now. They have no/little concept of time and I did not want to tell him that grandpa was sick and would die because I definitely did not want him thinking everytime you get a cold/stomach bug or whatever that you could die.
I also found it helped me deal with my grieve by explaining things to my son. It took some of the emotion out of it. Having him at the viewing also lightened the mood and helped my grandmother and others remember the youthful, carefree times.