My mom and grandparents only spoke Hungarian in the home until I was 5. My dad spoke English and a few words of Hungarian, so a very similar situation to your kids currently. We moved further away from my grandparents at that point so although I still spoke a lot, it became less and less as I grew older and assimilated into American culture.
The verbal and comprehension centers in the brain for language are separate. When a child is learning two languages simultaneously their comprehension centers are growing by leaps and bounds and the verbal centers not so much. The brain is actively trying to best understand and relate to all the words before deciding to work on the verbal. Hence, a seeming 'speech delay'. They are not delayed whatsoever as they are learning twice what monolingual kids are learning, and are learning it faster.
When you child is ready to speak he will do so flawlessly in two languages. I would totally discount the speech therapist unless they have specific experience in working with bilingual children. (most don't, trust me) I would also look into sending your kids to the bilingual school in DPS that teaches Mandarin (not the same as Cantonese, I know, but its also the culture, the characters, the customs and norms that are important to learn) http://www.denverchineseschool.org/index.php. Or at least consult with the teachers there who are used to seeing kids such as yours and can tell you more accurately what is 'normal' for your child in particular and what might be 'delayed'.
There is no greater gift that you can give your child than fluency in a 2nd language. Its tough to do, but sooooo worth it. Stick with it, stick with it, stick with it.