I get wanting to be neighborly and not have tension. You've made efforts to connect with his wife or with them as a couple, and that's fine. Originally getting some help from him and paying him was fine, but he can't handle it. If he had baked a bunch of stuff and just given you a little, that would be one thing. My neighbor (a single guy) has a bumper crop of cucumbers or zucchini from his garden now and then, and he leaves a bag on my doorstep and the doorsteps of a few others. But that's not what's happening to you.
The absolute red flags are the following: 1) He does it when his wife is away. 2) Even if she's home, she doesn't know about it. 3) Jewelry! Seriously? That is WAY too intimate a gift from a neighbor! 4) The big one that should cause you concern: he shows up where you are. Trust your instinct - this is stalking.
You sound like a person who tries to be very nice to everyone. You are shy and quiet, with some social anxiety, yet you push through that to be friendly. This is admirable. However, there are people you should not do that with. He's one of them.
I believe you should recalibrate your thought processes. You are more worried about his feelings and hers than about your own. You say you don't want to "come between them" but you don't see that YOU are not the one messing up their relationship - he is! This is all on him, or them - and not on you. Where you have made a mistake is in doing things for years that make you uncomfortable, rather than find a way out of it. You refuse, he insists, you relent, at least enough times that he keeps trying. And now he's cold and you feel responsible. I hope you can get to the point where you feel relieved. I do think you need to find a stock phrase or two, so that you let him know you are not comfortable with personal tokens from a married man. You can either return the jewelry to him (which may make him angry but will be a clear message) or you can give it away - but don't wear it. Instead of refusing heart-shaped treats because you're on a diet, you have to find a way to say, "You should give these hearts to your sweetheart, (wife's name). I insist! She's lovely and is the rightful recipient." If he pushes back, you say, "I insist. Gotta go. Bye now." Or, you can recall the old adage that goes: "No" is a complete sentence.
I agree with others that you hire neutral people to help you. You can go on Angie's List or Thumbtack or other services to find someone and check their reviews/references. If you are on Facebook, you can ask friends for recommendations. (If he's on Facebook and is "friends" with you, block him. If you can't afford all this outside help, then take a course at the bigger hardware stores on how to do some things, or barter your own services with another pro (just trade off what you can do for something they can do). You need an insured pro to clean gutters or do plumbing, but a resourceful teen can rake leaves and help you stack firewood or wash your deck. (I don't know what you need - these are just examples.)
If you want to continue being good neighbors, I would do things for his wife. Cook a meal and give half to her. Yes, he's going to eat it too, which makes it look like you're being friendly to both, but the favor is done for HER. It sounds like she doesn't get as much from him as she'd like. But I'd keep my distance overall, lest he see your gifts as outreach to him.
I hope you can find the strength to stand up for yourself and not feel the need to take on all kinds of anxiety and stress just to alleviate his. This could become an unsafe situation so stop him now.