But if defeat best reveals the nature of the man and his project, it is in success that his closest friendships and alliances have been forged. The first of these is his relationship with Steve Hilton, whom he first met in the late 1980s when both worked for what was then Conservative Central Office. Their decades-long conversation about the Tories, their strengths and weaknesses, their prejudices and favourites, is the dialogue that most drives the project. At first sight the two men could hardly be more different. Mr Cameron’s privileged background, social assurance and cultural Conservatism fitted him smoothly for the Conservative Research Department in the Tories’ old HQ in Smith Square. Mr Hilton, the son of Hungarian immigrants and a scholarship boy who became a Conservative only at university, was less obviously a CRD Tory boy. Mr Cameron favoured red braces; Mr Hilton was known to wear a “voluminous poncho”.
It is what they share, however, not what they do not, that most influences the modern Conservative Party. A passionate belief in the primacy of the individual over the collective – open in Mr Hilton’s case, partly shielded from view in that of Mr Cameron – is their first shared value. Mr Hilton, whose family endured the Communist repression of Budapest in 1956, has a visceral dislike of the statist mindset.
Neither, however, is a straightforward economic liberal. The purpose of freeing individual action is so that people can better deliver social goods. And the State has a role in fostering and encouraging those other institutions, such as marriage, that help people to share responsibility for one another. The pair have tried a number of attempts to rebrand these strands of right-wing philosophy during their period at the helm of the Conservative Party. It has been known variously as “modern, compassionate Conservatism”, the “post-bureaucratic age” and finally “the Big Society”. For the second big thing that Mr Cameron shares with his closest ally is an abiding interest in and facility for political communication. Both schooled according to the exacting standards of a Margaret Thatcher-era CRD, they write crisply and without jargon or cant. In the run-up to the 1992 general election the pair were selected to manage the relationship with the Tories’ advertising agency, Saatchi & Saatchi.
There is one final shared attribute: their age. They were part of a Smith Square “brat pack” in 1992 and were still young enough to weather the wilderness years so that they could emerge as part of the “next generation” just as new Labour was running out of steam.
In Stafford Leisure Centre in the early hours of May 2, 1997, however, those years outside power were just beginning. By then 31, the Conservative candidate, left hanging around as a loser at the count, knew that losing was no personal disgrace and certainly not a career-ending moment. Nonetheless, he felt it keenly when an elderly woman approached him in tears as the scale of the Tories’ national defeat became clear. “I don’t want to die under a Labour government,” she said. The misery of that exchange still lingered when he wrote of it many years later.
Mr Cameron has suffered personal setbacks. The influence of the birth, life and death, aged 6, of his first son Ivan is well known. It is fair to say, however, that he has suffered less in the way of professional reverses than many other senior members of the Conservative leadership. After defeat in Stafford Mr Cameron went back to a well-paid City job as director of communications for the media company Carlton. It was a younger generation of Conservative staffers who tasted the most bitter fruits of opposition, and started to do the most original thinking about how to return to power.
Although they overlapped while the Tories were in government, Mr Cameron did not really know George Osborne until both were elected in 2001. He was not in Smith Square as Mr Osborne and a handful of other young staffers, including several defectors from the Social Democratic Party, began to think deeply about how to decontaminate the Conservative brand. Figures such as Andrew Cooper and Rick Nye, who went on to set up the polling firm Populus, began pointing out that voters tended to like Conservative policies, until they found out that they were Conservative policies. It was not the policies that were the problem.
Just as Europe divided the party during its previous years in power, so the question of “de-toxifying” the Tory brand fractured it in opposition. The modernisation of the Tories, started under William Hague, slowed as it became entangled in the Tories’ kultur war over social issues such as gay adoption and marriage. Throughout it all Mr Hague’s political secretary, Mr Osborne, had a ringside seat.
Mr Cameron, as he now privately admits, had a slower conversion to the modernisers’ cause than some of his most senior allies. Less than a week after being elected in 2001 (having been selected for the safe seat of Witney) he was asked how the party should change. His answer is telling since it dwells on questions of presentation, not substance. “[The Conservative Party] needs to change its language, change its approach, start with a blank piece of paper and try to work out why our base of support is not broader. Anyone could have told the Labour Party in the 1980s how to become electable. It had to drop unilateral disarmament, punitive tax rises, wholesale nationalisation and unionisation. The question for the Conservative Party is far more difficult because there are no obvious areas of policy that need to be dropped.” Almost as an afterthought, he then added: “We need a clear, positive, engaging agenda on public services.”
Later, when he was leader, Mr Cameron was often asked when he would have a Clause Four moment, a reference to Tony Blair’s totemic defeat of party critics. His answer was always a version of that first, raw, draft. The riposte might be caricatured as: “We’re right, it’s just that the voters don’t realise it yet.”
Although he backed Mr Portillo, the modernisers’ candidate in the 2001 leadership election, the support was hesitant, even knowing. When Mr Portillo lost in an early round, the new Witney MP opted for Mr Duncan Smith over Kenneth Clarke. “What went wrong?” Mr Cameron mused in an online column. “Here was a leadership contender with buckets of charisma, a CV that included experience at the highest level of government and genuine cross-party appeal. Our man had offered leadership, radical change and ideas that challenged the party both in Parliament and the country. They simply weren’t ready for it. In many ways it is view that I share.”
Mr Cameron’s early career as a backbencher is not littered with examples of him acting as a spokesman for the need for the party to broaden its appeal. He was, for example, a passionate defender of fox-hunting. (In fairness, he also took a brave and principled position on the decriminalisation of drugs.) Gradually, however, and partly as a result of a developing friendship with Mr Osborne, Mr Cameron started to think more deeply about what was needed, and in particular what a “clear, positive, engaging agenda on public services” might look like.
At the same time, Mr Cameron was receiving firsthand experience of the NHS as it cared for his son, who was born with Ohtahara syndrome, a serious neurological condition. If the Tories were really going to modernise, Mr Cameron came to realise, they had to embrace properly funded, high-quality, universally available public services. Michael Gove, then a journalist with The Times, Mr Hilton, Mr Osborne and others began to meet regularly in a Mayfair restaurant to plot a Conservative future.
There was still time to test to destruction the alternative model. Mr Howard, who replaced the ousted Mr Duncan Smith in 2003, flirted with a full-throated modernisation but came to view it as unauthentic, at least in his mouth, and opted like Mr Hague for a safety-first “core-vote” strategy in the 2005 election.
Mr Cameron, who with Mr Gove helped to patch together the party’s manifesto, saw at first hand the consequences of limiting Tory appeal to existing supporters. Mr Cameron, then, emerged from another election disaster surrounded by two long-term friends, Mr Hilton and Mr Gove, and one newer ally, Mr Osborne. It was, however, someone he had known longer than any of them that pushed him hardest to run for the leadership. Andrew Feldman, a friend since Brasenose College, Oxford, set up the key meeting with Lord Harris of Peckham, a former Conservative treasurer, that helped to convince him to contest the leadership. Mr Feldman, who was later appointed chief executive of the party and then co-chairman, is an important, although non-political,