We went through this with our now 5 year old, and it did take a little while, but what we did was just keep telling him those are words that adults use. We let him know it was normal for people to say them, but that only when you're an adult. We didn't actually punish him when he would say one, but we'd talk to him and remind him he was using an adult word and he was not an adult yet.
The other thing that I was told to try, that actually seemed to work also, was letting him know that if he really felt the need to try saying those words he could do it in his bedroom when he was alone. That way we weren't telling him he absolutely couldn't do it - which would just encourage him to do it more - but we limited the times and places when he could say it. We did tell him that we did not want to hear those words, so if he chose to say them he had to go to his room and be by himself (meaning he couldn't just say them anywhere in the house, or if anyone was with him). There were a few times he let me know he had said the F word when he was in his room (and I let him know that was ok), but overall he got through this phase without too much trouble and we didn't have any problems with him swearing when he started preschool.
Now he's aware of what "naughty" words are and who is allowed to say them. He even lets us (and others) know when someone says one, but we haven't had any recent problems with him feeling the need to use them himself.
Good luck!