Can you keep your bedroom door closed to keep the cat out? If so, try placing a clean litter box outside the door, and if the cat uses it, gradually move it, a few feet at a time, to the closet you prefer. Or perhaps there's another location between those two that would actually work better?
Nature's Miracle, a cleaner/deodorizer available at pet stores, is reported to work wonders on animal smells. Good luck!
ADDED: another answer reminded me of a cure a friend used for discouraging a cat from crossing a no-no boundary. She laid down packaging tape, sticky side up (tape the ends down to keep it in place). Cats detest the feel of that. You could also apply the tape to a large piece of cardboard to lay over the spot. I'd make it as big as possible, or several smaller pieces, so the cat won't just go right next to it.