We love the Curious George soundtrack. We also have a couple of They Might Be Giants CD's and one or two by Dan Zanes that are fun.
My daughter loves to hear her own voice, so I gave her my mini cassette recorder and showed her how to play and rewind (only I get to push the red button to record -- for the safekeeping of the precious recordings!). If you're able to record your son singing his favorite songs and then burn them onto a CD that he could play on his little player, that would be so much fun. He could sing along with himself!