My daughter is much the same way. (She is 5 now.) She's just now enjoying the bouncy houses and I'm not sure she'd go on them each and every time, she's still pretty situation-specific.
What we did was just not make a big deal out of it. She got no attention either way from us; we didn't beg, we didn't get upset, we just said, "ok" and walked away with her and watched her big brother do things.
I think that you have a similar problem to what we had, which is not so much the *activity* but the factors surrounding it. My daughter has a terrible time sorting out things when there is a lot of commotion or a lot of people in one place. She will break down and cry with what has seemed like fear but we have actually realized in her case she's just in sensory overload. The noise and activity is too much. I've found that preparing her for things, such as saying "this will be noisy, but it's ok because we'll be there together" has helped a lot. She doesn't have any big or terrible sensory issues, she just doesn't deal well with that sort of thing. (It's certainly not a volume factor, since she dances to Hannah Montana full blast LOL)
edited to add: I just remembered, we had this issue last Monday. We went to a little soccer class and there were a ton of kids. She refused to do anything, said she was scared, and we ended up leaving. I'm sure that it was the sheer number of kids and that she was overwhelmed. (We ended up leaving because of her standing there shrieking "I can't do this!" in a very dramatic way.) Other parents think they're helping by giving a lot of attention to it and begging her to join, but that freaks her out more. (Hence the screaming.)
Good luck and my advice is to let it ride for now. She's still such a young one, SHE is following her instincts and not the crowd. :)