Sadiq Khan wins London mayoral election: Where Zac Goldsmith went wrong

On paper Zac Goldsmith had everything going for him. Good-looking and charming, with proven cross-party appeal and a record of independence from the Tory groupthink, Labour feared he would be a formidable opponent in the race for London mayor.

The bookies agreed. Ladbrokes initially rated Goldsmith as the clear favourite to succeed Boris Johnson, with Sadiq Khan trailing far behind. One early YouGov poll put the two men neck-and-neck in the race for City Hall. With the company behind Boris Johnson's two campaign wins hired again, the Tories believed they had a fighting chance of hanging on to London. Some within Labour also had private doubts about their candidate. "Sadiq can't win," one senior London Labour politician told me shortly after both men were nominated.

Yet fast forward seven months and Goldsmith suffered a crushing defeat by Khan, after running what one Conservative-supporting magazine described as a "toe-curling embarrassment" of a campaign.

So what went wrong for Goldsmith? How did the man with everything going for him end up being steamrollered by the Khan machine?

The first point to note is that despite those early expectations, Goldsmith always had a mountain to climb in London. Despite losing two mayoral elections in a row to Boris, Labour has been quietly progressing in the city for years, making big gains both at last year's general election and at the last London Assembly and local council elections. The last two mayoral elections hid this imbalance because of the relative prominence of both Livingstone and Johnson. To the man in the street, Goldsmith and Khan were unknowns. So with everything else being equal, the Conservative candidate was likely to lose and the Labour candidate was likely to win.

As long as Khan held onto Labour's vote in the city he was the likely winner. The challenge facing the Tories was to buck that trend. In order to win, they had to reach well beyond their core vote and pick up the second preference votes of Green and Lib Dem voters. On paper, Goldsmith had the potential to do that, given his long record campaigning on environmental and democratic issues. Yet rather than emphasise that, the Goldsmith campaign chose to pursue a core vote strategy targeted almost exclusively on targeting Outer London Tories.

Whereas at the start of the campaign there was widespread indifference to Khan's candidacy among many Labour supporters, the viciousness of Goldsmith's campaign undoubtedly rallied a lot of Labour activists to come to his defence.

Having won two mayoral elections by boosting Tory votes in the suburbs while depressing Labour votes in inner London, Goldsmith's campaign calculated that they could make it a hat-trick. This was the so-called 'doughnut strategy'. The main problem with this strategy was that recent demographic changes in outer London mean it is no longer anywhere near as solidly Conservative as it once was. In 2012, Ken actually won in most parts of London, but was beaten thanks to large numbers of Tory votes in Outer South London. Yet the last YouGov poll found Khan and Goldsmith were almost neck-and-neck in the outer boroughs. The Tories' beloved doughnut has crumbled.

The second major problem with the Tories' strategy was that Goldsmith's relentlessly negative and racially-charged campaign has backfired. Some London Tories were privately angry about the dog-whistling anti-Muslim nature of Goldsmith's operation, having worked hard for many years to improve the party's support among ethnic minorities in the capital. Last night, the former leader of the London Assembly Tories told Newsnight that Goldsmith had done "real damage" to the party in London, particularly among Muslims.

Sources close to Khan also believe the attacks actually motivated a section of their own core vote to come out to stop Goldsmith. Whereas at the start of the campaign there was widespread indifference to Khan's candidacy among many Labour supporters, the viciousness of Goldsmith's campaign undoubtedly rallied a lot of Labour activists to come to his defence.

The attacks also undid the main appeals of a Goldsmith candidacy, which was his cross-party appeal and his record of independence. By opening himself to charges of racism, the Tory campaign made Goldsmith toxic to supporters of other political parties. And by forcing him to stand shoulder to shoulder with David Cameron and George Osborne, he undid any impression that he would be a mayor willing to stand up to his own government.

If Londoners already had a view of him as an out of touch millionaire dilettante, then he did little during this campaign to dispel that impression.

The third and final problem with the Tory campaign was Goldsmith himself. Khan may not be the most charismatic politician, but he at least gave the appearance of wanting the job he was running for. At times Goldsmith looked like a man who had been held hostage by his own campaign. Lacklustre debate performances were compounded by his general laid-back and patrician manner.

Earlier this week I watched Goldsmith walk around market stalls in the white working class area of Erith. With his softly spoken voice and hand in his side pocket, Goldsmith had the air of a minor royal on a visit to an inner city youth centre. This otherworldliness was confirmed in multiple disastrous television interviews – firstly with the BBC in the back of a taxi where he failed a quiz about London, and secondly at the Asian Awards– where he boasted of his knowledge of Bollywood films but was unable to name a single one. On both occasions he looked awkward, uncomfortable and ever so slightly dim. If Londoners already had a view of him as an out-of-touch millionaire dilettante, he did little during this campaign to dispel that impression.

This was always going to be a difficult race for the Conservatives to win but it was not impossible. A recent poll showed that in a race between Khan and the outgoing Conservative mayor, Johnson would have narrowly won. Had Goldsmith fought a better campaign, he could have been in with a fighting chance. Instead he has delivered the party a humiliating and potentially damaging defeat.

Adam Bienkov is the Deputy Editor of Politics.co.uk