You've gotten some really good answers! The fact that Matt has hung around for a year, getting to know you and your kids without having been a parent before, speaks volumes for him! I think that letting he and Charlie have some alone time is an excellent idea and the fact that he suggested an activity that takes energy means he somewhat understands Charlie.
I, too, have a child with issues - Tourette Syndrome, ADD, mild OCD, depression - and can totally understand your urge to intervene. But, you have to understand that Charlie needs to learn his limits with Matt, AND he may act totally different without you around.
When I was dating, I *thought* my then boyfriend liked my son - they seemed to get along well and my son needed a father in his life; he was not quite 2 years old at the time and had never had a dad. My boyfriend had never had children, either, but he also didn't do anything alone with my son to try to bond with him without me (nor did he even suggest it). When we discovered my son's issues, we tried it all - allergy shots, medications, counselors, etc. And yes, there were times when even I didn't want to be around my son, so I understood why my BF didn't want to be around him. My boyfriend (who I married) didn't follow through on what the counselors suggested and didn't believe that my son's behavior was caused by his issues. I was in the middle of the two of them constantly, and rightly so because my husband treated him very badly - I really should have left. To this day I feel extremely guilty because I married someone who hated my son from day one (he admitted it to both of us once my son was grown).
So, all this to say: let Matt and Charlie bond, talk to Matt about your feelings and about Matt getting educated on ADHD, see how it goes, but DON'T make any commitments until you see how it's going to go. Communication between the two of you is very key and if Matt is willing to try, then I would certainly give him a chance. Charlie is going to have a lot of interactions with a lot of people during his life, and he needs to start learning how to handle it now and how to manage his ADHD and not blame everything on it (which is what my son does). Hopefully I haven't discouraged you, because that is not my intention. Best wishes and I hope all goes well!!!