J.F.
We took our family to NYC when our youngest was 5 1/2, and all of us, from the youngest to the oldest, had a great time!
We went to the Statue of Liberty and toured Ellis Island; got sandwiches at Carnegie and ate in Central Park, then went for a carriage ride around the park; went to 3 Broadway plays---with the 5 year old (he did great and loved them); walked around Chinatown and Little Italy; toured NBC studios; went to the WTC site and the church near the site; had private tour of NY Stock Exchange; toured Federal Hall; went to Battery Park and walked around surrounding area.
Our little one also loved FAO Schwartz, and that is one thing you can do if the weather isn't cooperating. Another big highlight for him was meeting "Spiderman" on the street and getting his picture taken with him while we were just out walking around. He really had fun everywhere we went, and it was a great opportunity to teach him about history, commerce, and so many different cultures in a way that he could understand.
That was right before he started Kindergarten (about a year and a half ago), and he still talks about the trip and all we did. This year, he was informing his first grade class about Central Park, and his memories were so vivid, his teacher couldn't wait to tell me about his story and all of his animated details. So, even though your son is young, he will still appreciate and enjoy all that the city has to offer.
I don't know of any festivals during that time, but I'm sure you could google and find out.
Whatever you do, I'm sure you will have a great time, and it will be a wonderful experience that will make wonderful memories for your son as well as for you and your husband!
Hope you get some decent weather. Enjoy!
J. F.