Mr Khan told BBC Radio 4’s Why Do You Hate Me? podcast that the words on the recording sounded similar to his voice.
Far-right groups picked up on the footage and it led to offensive online comments against Mr Khan.
Mr Khan, in an interview with the BBC, said: “When you’ve got friends and family who see this stuff, it’s deeply upsetting. I mean, I’ve got two daughters, a wife, I’ve got, you know, siblings. I’ve got a mum.”
“The timing couldn’t have been better if you’re seeking to sow disharmony and cause problems,” the London mayor told the BBC.
“What was being said was a red rag to a bull for the far right and others. But what concerned me the most was if you’re an innocent listener of this. Because it’s a secret undercover recording, in inverted commas, because it sounds like me, because of the timing and the context.”