What a good question. Here's what I think.
He just seems really uncomfortable and that makes the audience feel uncomfortable. He's also quite slow moving about the floor and, as he's already demonstrated himself, he doesn't know what to do with his hands, which just adds to the uncomfortable feel. Plus, for this show, he's up against the memory of Russell Brand who could think faster on his feet than anyone else I've seen on TV. Compare Chris's rundown of the contestants and think what Russell could've done.
Great on the radio, though.