The questions as I ask myself when one of my girls is hurt are these:
1) Did I cause the pain to the child?
2) Was I around when it happened (i.e. in the same room)?
3) Could I have stopped it?
If I say yes to any of those - then I feel bad. However, if I answer no to all three then I work through and erase the guilt. If I remember the situation, you were not in the room and could not have stopped it, and because you weren't in the room - you obviously did not cause the pain. So therefore, you shouldn't feel guilty. I know it's hard. My oldest daughter recently pushed her sister down a couple stairs - she now has a HUGE shiner and on top of the bruising she is all scratched up. I wasn't home, my husband was in another room - neither one of us did anything wrong. However, every time I see her I feel a little guilt (her sister feels a lot more). Give it time and you'll work through it.
As far as people staring, let them stare - maybe they're thinking the same thing that you do if you see a little one with a cast on.