We stayed at Steamboat Inn w/our then 2 yo for a long weekend. We got the biggest suite, w/a separate living area that can double as bedroom. It has a little kitchenette and the biggest bathtub I've ever seen. I like the 3pm teatime, where you can get an assortment of cookies and pastries along w/coffees, teas and sherry in the upstairs dining area. Its a beautiful little hotel right on the river - from our room you could see the bridge. There is also a little park across the river from Steamboat that we let our son play soccer.
We didn't eat at Mystic Pizza (all the locals said it was bad pizza) but we ate at every other pizza place and they were really good. Our favorite was Capt Daniel Packards restaurant. Best lobster ravioli. Pricey but good upscale food. They had a high chair for our 2 yo so they were accommodating and our 2 yo did fine (he's good out in public). CDP also has a pub w/a pub menu downstairs, if you want something a little more relaxed.
I second everything everyone suggested - Pequot, Old Mystic Village for some cute shopping and the Aquarium. Mystic is also known for Black Dog goods, so try to go to their store off Main Street.
Have fun! We love Mystic and hope to go back someday.