We've taken our kids camping at that age. Along with the camping gear and the usual things, we added a pack n play. We usually go to a campground that has a pool (and sometimes a lake or pond for fishing) and other activities. My friend took her kids to a campground that was on the beach. They could enjoy all of the amenities of the campground as well as the beach.
Several of my friends have taken their kids to the beach. Their children range in age from newborn to age 5. Just remember the sunscreen.
I can't think of many places I would be unable to take my kids. Think about what sounds fun to you. Then think about what you would need to do to include your son. Hiking, for instance, could be a great way to spend time. To include your son, grab a baby carrier like the Ergo carrier where you can wear him in a front carry or on your back. Additionally, he could be worn by either you or your husband. He is old enough to take in a pool or onto a beach as long as you watch him. If you want to go to either and want a little more freedom to relax, take a pack n play and some toys for in it. He can play with you and while you get a little tanning or reading in play in the pack n play.
I know that a lot of places may sound daunting. But even sightseeing is something you can do. Pick a place you want to visit (if we had the money, we'd be going back to visit New Orleans). Your son is still at that age where they usually like to check out their surroundings. Isn't that what you are doing when you are sightseeing.
The only thing that I would be concerned about is a long drive if you son doesn't like being in his carseat. We've done long drives with our kids without problems. We just planned lots of stops and did a great deal of our driving after our kids bedtimes letting them sleep in their carseats,.
I mentioned going back to visit New Orleans. We did that with our daughter when she was 3. They have a nice zoo, a great acquarium, some wonderful playgrounds, many sights to see, many tours you can take, and wonderful wonderful food.
Find a place with sights you want to see and pack a few toys for your son. One of the keys is to try to keep some semblance of his schedule while you are on vacation. If a certain time of day is his naptime, try to keep that as his naptime while you are on vacation. Maybe not everyday, but most days. Also, try to keep his feeding schedule the same. These things will help keep him happier, well rested and well fed. Also, plan a little time each day to just sit and play with him. For instance, if you stay at a hotel/motel, set aside a half hour or so where you just sit in the room and play with his things in the morning and maybe another half hour at night just to help him unwind.
Anyhow, I guess what I am saying is go where you want to go. There are some things he won't be able to do (skydiving would be out!) but there are so many things that he can just be a little tag along with you on. The key is knowing how you want to manage it. Are you going someplace where you will use a stroller all the time or would you rather bring your (or get if you don't have one) baby carrier with you? Do you need your pack n play or will there be a crib at the place you are going to?
Also, another great option is the grandparent option. Pick someplace that is near relatives (your parents or your husbands for example) and see if they will keep your son for part of the time so that you can do what you want to do for a couple days. Then pick up your son and take him to do those things that you think he would enjoy coming along for. You might even want to invite the family members to join you for a day or two.
Anyhow, I hope this helps. Good luck and hope you have a great vacation.