An empire of information, in some form, existed at home too. New public health initiatives and social programmes meant the state knew much more about its people than ever before. Meanwhile Irish nationalists and bomb-throwing anarchists from the Continent had forced the Metropolitan Police to develop the world’s first ‘Special Branch’ in 1883. Inevitably perhaps, by the first decade of the twentieth century, Special Branch had broadened its net beyond Fenians and anarchists to collect intelligence on possible foreign spies, anti-colonial agitators, suffragettes, union leaders, pacifists, and even those with radical views on sex.5
Nevertheless, these developments were more about keeping authoritarian surveillance at arm’s length from the British public. Mysterious battles with Russian spies in Central Asia were reassuringly remote, while the creation of a small Special Branch of intrepid detectives to counter terrorism was seen as a low-key alternative to the sort of harsh and repressive security legislation preferred in Europe. Accordingly, intelligence remained a matter for distant vice-consuls travelling to Samarkand, or detective sergeants in Whitechapel. It rarely resonated at the centre of British government. More rarely still did a prime minister take a personal interest in espionage.
All this began to change over the next decade. Bizarrely, it was literary fashion which drove the transformation of British intelligence, thrusting it towards the heart of government for the first time. A wave of popular Edwardian ‘invasion literature’ forced Herbert Asquith, an otherwise uninterested Liberal prime minister more focused on progressive domestic reforms, to take notice of foreign espionage. Acting only under pressure, Asquith set in train a course which would fundamentally alter the landscape of British intelligence forever. The intelligence revolution may have been forged by Churchill and Attlee, but the first sparks came through the unlikely Asquith. Nonetheless, despite the expansion of espionage during the First World War and greater interest on the part of Asquith’s successor David Lloyd George, it took time and endless trouble before successive prime ministers learned to use intelligence effectively.
Fiction paved the long pathway to the creation of a modern British intelligence service. During the nineteenth century, a stream of often sensational crime novels fed a growing public fascination with detection, police work and science. This generated cultural change, gradually eroding anxieties about government surveillance. Created by writers as diverse as Edgar Allan Poe and Arthur Conan Doyle, fictional detectives gradually became heroic figures to be worshipped, rather than bogeymen associated with distasteful and un-British authoritarianism.6 Journalists, who often exchanged information with real policemen, accelerated this transformation, helping to create real-life detectives who enjoyed a similar cult status.7 William Melville, who became head of Special Branch, was one example. He famously foiled several anarchist attacks, including the ‘Jubilee plot’ against Queen Victoria in 1887 and another dastardly scheme known as the ‘Walsall plot’, in which anarchists from the West Midlands sought to manufacture a bomb. Many now believe that he created the latter plot himself for mere self-glorification.8
Over three hundred spy novels went into print between 1901 and 1914, mostly focused on the ‘German Menace’. This new fiction effectively rebranded itself as a barely concealed form of ‘true crime’ writing, supposedly based on the patriotic leaking of government secrets. In 1903, Erskine Childers published The Riddle of the Sands, which narrated the summer sailing adventures of two young British men along the East Frisian coast who stumble upon a German plot to invade England with a flotilla of barges. Childers claimed to be offering the British people nothing less than a warning of what was to come, and Rudyard Kipling urged the public to support him in taking a firm stand against the ‘shameless Hun’.9 The story was improbable – the German admiralty had long written off the possibility of an invasion from East Frisia – but the hapless Royal Navy was forced to investigate the claims regardless.10
Members of Parliament from constituencies on the east coast, the closest part of Britain to Germany, bombarded the First Lord of the Admiralty, Lord Selborne, with difficult questions. In response, he ordered the Naval Intelligence Division to write a detailed report on the invasion plan outlined by Childers. After reconnaissance of the Frisian coast, the report concluded that the lack of railways and roads, together with the shallowness of the water and general lack of facilities, made a secret invasion from there impossible. ‘As a novel it is excellent; as a war plan it is rubbish,’ insisted Lord Louis Battenberg, the director of naval intelligence. To Childers’ barely disguised delight, the Kaiser banned the book in Germany. Childers also claimed that when he next went sailing in the Baltic, German spies dogged his every move.11
Spy fiction developed a darker side, shifting its focus closer to home, portraying immigrants and foreign visitors as the hidden hand of subversion. The new trope was Germans as a hidden fifth column already within Britain. Most active of all was a mischievous thriller-writer called William le Queux. He almost single-handedly created a fifth-column panic and then demanded the creation of a domestic security service to combat the undercover ‘German Menace’, with which the day of reckoning would surely come.12 Le Queux’s The Invasion of 1910 became the most influential of these books. Published in 1906, it told of German armies over-running the country with the help of spies and saboteurs. Le Queux admitted to a political purpose, explaining his intention to illustrate how poorly British defences would stand up to this sort of sneak attack. He also insisted that the content of his book was factually correct, and had been informed by conversations with ‘the authorities’.13
William le Queux posed as an international espionage expert, but was in fact nothing of the sort. Instead, he was a tireless and well-paid pulp-fiction writer who produced five novels a year until his death in 1927.14 The Invasion of 1910 sold more than a million copies, and publicists at the Daily Mail, which possessed serialisation rights, sent columns of men marching up Oxford Street dressed in Prussian uniforms, complete with bloodstained gloves, carrying sandwich boards that advertised the book.15 An inevitable flood of literary emulators hit the presses as le Queux’s royalties rolled in. Not to be outdone, le Queux responded in 1909 with Spies of the Kaiser, which insisted that no fewer than 5,000 German undercover operatives were at work in Britain. This was a new idea. A whole undercover army, not just a few spies and saboteurs, were apparently biding their time until the Fatherland told them to retrieve their weapons from a series of arms dumps in the British countryside. No less important, the novel also claimed to offer the inside story of intrepid British detectives working with agents to uncover these foreign networks.