D.B.
Do you mean Montego Bay? I never heard of a Monteray Bay there.
Due to the crime situation, I would stick with crowds and supervised things. This is not the time to roam around into local areas. Montego Bay is a big tourist site so this is not where you'll see a lot of "local flavor."
Dunn's River Falls is a fantastic trip but it's a ways from Montego Bay. I'm not sure how tight your connections are for that day. If you can get an official tour bus from the port along the north shore to the falls, take the tour with a guide (they are awesome, and you need an expert to help you walk up the falls safely), and get back in plenty of time, then fine. I'm not sure this is what you mean by "relaxing" but it's a natural wonder. But be sure you leave a big window of time for delays and traffic and so on.
I would talk to the cruise line about options. If you want another opinion, I'd talk to a good travel agent (there aren't so many around, but if you're a AAA member, you can get good and current info from their travel department even if you aren't booking your travel through them. It's a member benefit.
Otherwise, I would just shop a little and go back to the ship to relax. If most people are on the island, the pool and other features on the ship will be lightly used.