Beyond Bletchley: GCHQ and British Intelligence

This year, the UK’s signals intelligence, cyber and security agency, the Government Communications Headquarters (GCHQ), marks its centenary. Formed in 1919, GCHQ – the largest, yet least understood, of Britain’s intelligence agencies – has been at the forefront of security. But its history has been wrapped up with that of Bletchley Park, the Second World War home of the Government Code & Cipher School (GC&CS), located in the Buckinghamshire countryside, which is now most famous for the role played by the mathematician and computer pioneer Alan Turing.

The breaking of Germany’s wartime codes, especially the decryption of the electro-mechanical cipher machine Enigma, was described by the historian David Kahn as the ‘greatest secret of World War II after the atom bomb’. First revealed in the 1970s by former insider F.W. Winterbotham, Bletchley’s work – and the experiences of those who worked there – have become engrained in popular culture and public history. For some, Bletchley was decisive in winning the war. Without the intelligence (codenamed ULTRA), Britain – and its allies – would have lost, or so the story goes. The official historian of wartime intelligence Harry Hinsley even went so far as to suggest that Bletchley shortened the war by two to four years. Yet the ‘Bletchley myth’ is overstated. Useful at the operational and tactical level, Bletchley’s intelligence was just one reason for Allied success. Contrary to popular perception, the intelligence was far from perfect and often misinterpreted. Intelligence seldom wins battles itself; equally important is the application of technology, resources and those on the front line.

Most secret

The focus on its wartime work is both a blessing and a curse for GCHQ and British intelligence history. Just as the foreign intelligence agency, the Secret Intelligence Service (MI6), is unable to escape the legacy of James Bond, GCHQ remains wedded to its wartime predecessor. Yet, while few historians would leave out ULTRA from its achievements, many forget GCHQ’s peacetime work.

GCHQ has been considered Britain’s most secret agency. Only avowed in 1982, the organisation was placed on the statute books (alongside its better known sibling SIS) by the Intelligence Services Act in 1994. With its headquarters based in the ‘Doughnut’, near Cheltenham, GCHQ employs around 6,000 staff at several known sites around the UK: in Manchester, Bude, Scarborough, Harrogate and London. GCHQ also created the National Cyber Security Centre (NCSC), formed in 2016, to work with the wider public sector, industry, small businesses and the public, protecting critical national infrastructure, while warning about day-to-day cyber threats. It is the largest of the UK’s agencies, soaking up much of the intelligence budget, even if SIS and the UK’s domestic security agency, the Security Service (MI5), often attract more attention. While the public fixate on spies and special operations, it is GCHQ that provides much of the secret intelligence for government, collecting signals intelligence – the interception of electronic signals, known as SIGINT. This can be divided into COMINT, or communications intelligence, information from phone calls, texts and digital messages; and ELINT, electrical intelligence from radar emissions and other signals. GCHQ’s other job has been to protect the UK’s communications – in effect, stopping other countries stealing British secrets, through the Communications Electronic Security Group and its predecessors.

GCHQ now has a greater public profile, as its centenary shows. It is on Twitter and Instagram, and seeks to attract a new generation of tech-driven recruits. It has a special Science Museum exhibition dedicated to its work, ‘Top Secret: From Ciphers to Cyber Security’, and next year sees the publication of an authorised history (joining SIS and MI5) by John Ferris. Its director, Jeremy Fleming, makes regular speeches and the organisation wins awards for its LGBT and inclusivity agenda. Like MI5, it releases records to the UK National Archives. Such activity was almost impossible in the ‘culture of secrecy’ that shrouded much of Britain’s agencies into the 1990s. But the study of GCHQ and its predecessors and its impact on foreign and security policy in the 20th century remains limited. Newly declassified material, official history and former insiders means that intelligence has long stopped being the ‘missing dimension’ of Britain’s political and military history, but spy stories and tales of daring exploits still dominate. Although much of GCHQ’s post-1945 history is closed off to researchers – a gap that will be filled by its new authorised history – its work before the war is little understood, too.

State secrets

The making and breaking of codes and ciphers has always been important. The Elizabethan state had its own cipher bureau, notably revealing plots against the sovereign. Later, the Post Office had its own ‘Secret Department’, opening the mail of suspected agents, which was abolished in the 1840s as a result of political scandal. But the origins of GCHQ can be traced to the early 20th century and the formal creation of the UK’s intelligence agencies. From 1909 Britain had a small Secret Service Bureau, responsible for catching spies and running agents, eventually becoming MI5 and SIS. But few had actually thought about exploiting the spread in wireless technology at the start of the century, with the exception of a tiny army team (just two people) in India, attacking Russian, Chinese and Tibetan messages. Early SIGINT was haphazard and it was only with the First World War that Britain started the process that led to the establishment of GCHQ. By September 1914 army codebreakers had begun their own operations under the War Office, known as MI1(b). The Admiralty had set up its own separate organisation, later christened Room 40, based in the Old Admiralty Building, Whitehall, initially supporting fleet operations in the North Sea. Both enjoyed successes; MI1(b) conducted diplomatic cryptanalysis, while the Admiralty, by intercepting Germany’s naval messages, was able to warn of surprise attacks on the Grand Fleet. In 1917 it was the publication of the secret Zimmermann Telegram – first intercepted and decoded by Room 40 – that convinced the US government to declare war on Germany, tipping the balance of power in favour of the Allies.

SIGINT had played a valuable role during the First World War, but there were significant disagreements over who should control it. The War Office wanted their own man, Major Malcolm Hay; the Admiralty, Alistair Denniston. In the end, codebreaking responsibilities fell to Denniston and a new organisation, the Government Code & Cypher School (GC&CS), emerged on 1 November 1919, placed under Foreign Office control in 1922. In public, its remit was to protect Britain’s codes and ciphers, stopping others from stealing information. Its private role was to ‘study the methods of cipher communications used by foreign powers’. Compared with GCHQ now, it was small, having just 50 staff, rising to 200 by the outbreak of the Second World War. In 1921, the Foreign Secretary, Lord Curzon, wrote that the department was ‘the most important branch of our confidential work’, offering the ‘most accurate and … the cheapest, means of obtaining secret political information’. GC&CS had significant early success against French, Japanese and US communications. The main target was the Soviet Union’s messages, thanks to government fears of revolution and subversion at home and in the Empire. In the 1920s GC&CS’s success against Soviet communications gave officials and ministers unrivalled insights into Moscow’s foreign policy, yet Britain’s most important source was compromised by ministerial leaks, first in 1920 and then in 1927, trying to justify Britain’s hard-line anti-Bolshevik stance. Having improved its security, Moscow became – to quote Churchill – ‘a riddle, wrapped in a mystery, inside an enigma’. By 1939, GC&CS had turned its focus – helped by French and Polish codebreakers – to Nazi Germany.

The end of the Second World War led to change. GC&CS had a good war: its work at Bletchley shaped military operations, while its commercial and diplomatic work (located for much of the war in London) played an important role in the economic and political war against the Axis. But the emergence of the Soviet threat and the start of the Cold War would see a new peacetime organisation. By the end of 1944, there were over 10,000 staff at Bletchley and GC&CS’s outstations. A year later there were just under 2,000, still far bigger than the interwar GC&CS. On 1 April 1946 it moved from its wartime home to a new site at Eastcote, Middlesex, later moving to Cheltenham (where GCHQ’s headquarters would stay) in the 1950s. It was also officially renamed the Government Communications Headquarters, a cover name used as early as 1939. Unlike its wartime successes, GCHQ’s Cold War would be frustrating. With a few exceptions, work against the Soviet Union’s codes had stopped in wartime, only resuming in July 1945. Initially British codebreakers and their US allies enjoyed considerable success against Soviet systems, providing information on the Soviet military. Low level reports on Eastern Bloc targets were also giving insights into the political, economic and security activity behind the ‘Iron Curtain’. But breaks into high-level Soviet codes sharply declined after Friday 29 October 1948 (or ‘Black Friday’), when the Russians changed virtually all their cipher systems, tipped off by leaks from the US codebreaker and Soviet spy William ‘Bill’ Weisband and the SIS officer turned KGB agent Kim Philby. A 1951 review of Britain’s agencies concluded that, while GCHQ provided as much as 90 per cent of secret intelligence on the Soviets, there were no breaks into ‘high-grade Russian traffic’, though other methods such as traffic analysis were reaping rewards.

Success stories

There were successes. One particularly important breakthrough was against wartime KGB traffic, codenamed VENONA. For Western intelligence, the breaks into KGB communications gave clear evidence that Soviet intelligence had penetrated the West’s atomic bomb programme, government and even intelligence agencies. SIGINT success led to the unmasking of the atom bomb spy Klaus Fuchs, sentenced to 14 years in prison in 1950. VENONA was also behind the unmasking of the Foreign Office official Donald Maclean, ultimately leading to his defection in May 1951. But VENONA’s successes had their limitations; like all SIGINT sources, it remained a closely guarded secret, only officially revealed in 1995, with its evidence never used in court. GCHQ’s effort also used special ELINT flights – aircraft known as ‘Ferrets’ – to build up a picture of Soviet radar and missile sites. While GCHQ’s Cold War was less dramatic than that of SIS, the gathering of information on the Soviet military was vital in reassuring policymakers that a Third World War was not going to break out. Attacks on ‘softer’ non-Soviet targets were also important in giving glimpses into Moscow’s thinking, as were joint US-UK operations (codenamed BROADSIDE and TRYST) listening secretly to the Soviet leadership’s calls from their Chaika limousines in downtown Moscow.

Beyond the main Soviet target, GCHQ also enjoyed successes against smaller foes – though much of its postwar diplomatic eavesdropping remains secret. In the 1950s, the Foreign Secretary Selwyn Lloyd thanked Cheltenham for its ‘valuable’ work against countries in the Middle East, while, during the Falklands War, the Labour MP Ted Rowlands revealed sensationally to the House of Commons that GCHQ read Argentine diplomatic messages – a leak just as damaging as those of the 1920s. In 2003, a whistleblower, Katherine Gunn, leaked a request from US allies to eavesdrop on the UN offices of Angola, Bulgaria, Cameroon, Chile, Guinea and Pakistan. Leaks by the former US National Security Agency contractor Edward Snowden – an event that did untold damage to US-UK SIGINT – also reveal UK and US successes against allies, neutrals and enemies alike.

Special relationship

GCHQ’s history is more than just spy stories and secret intelligence. For much of its existence, it has been at the forefront of the UK’s diplomatic efforts in cementing the ‘special relationship’ with the United States and its Commonwealth partners. It has been intelligence that puts the ‘special’ into the ‘special relationship’, whatever political differences exist between presidents and prime ministers – even today. For all the talk about Britain’s decline as a major power post-1945, intelligence is one of the more important things Britain brings to the table for the US. Though today’s ‘Five Eyes’ or ‘UKUSA’ alliance – comprising the US, UK, Canada, Australia and New Zealand – is a Cold War construction, its origins go back to the early days of the Second World War and the growing partnerships that emerged in wartime. As transatlantic allies turned to the Soviets, liaison became essential. This postwar SIGINT partnership was cemented in Washington on 5 March 1946; ironically, the same day Churchill delivered his ‘Iron Curtain’ speech arguing for a ‘special relationship’. The document, the ‘British-U.S. Communication Intelligence Agreement’ (or BRUSA), declassified as recently as 2010, represents a landmark moment for transatlantic relations. Future liaison would expand to form the ‘Five Eyes’ community, including the Commonwealth partners.

Working with transatlantic counterparts produced a close bond. While prime ministers and presidents would often disagree, codebreakers would continue their work together. GCHQ’s liaison with the US survived the humiliation of the Suez Crisis. Similarly, in the early 1970s, Britain’s pro-European policies led President Nixon to threaten intelligence ties. Britain introduced its own restrictions, yet SIGINT cooperation remained intact: both sides knew its importance. One reason for the enduring relationship was the mutual threat of the Soviet Union, though the ‘special relationship’ ran deeper. In July 1969, GCHQ’s director Leonard Hooper told his US counterpart: ‘I have often felt closer to you than to most of my own staff.’ Such liaison led to what the former GCHQ official Michael Herman refers to as a professional community with standardised security arrangements and techniques. On a personal level, the special relationship is alive in the chess clubs, sporting events and even relationships that sprang up after years of working together. Though some have questioned whether the special relationship is real or rhetorical, it is alive and well in the intelligence world.

In SIGINT, geography matters. Understanding the form of communication you want to attack – short-wave radio messages, satellite communications, or even today’s fibre optic cables at the heart of global communications – means you need to be in the right place at the right time. For much of its existence, signals were collected through a network of ‘Y’ stations, a series of intercept sites scattered across Britain and its Empire, staffed by thousands of young men and women. Many were drawn from the armed forces, intercepting and transcribing messages and passing the raw material to those who would then break it down. Sites at Beaumanor Hall, RAF Cheadle, RAF Chicksands, Cromer, HMS Flowerdown, Goonhavern, Hawklaw and Tean played a vital role. Today, only one, GCHQ’s Scarborough facility, has been operational throughout its history (predating GC&CS’s formation as an Admiralty Wireless Telegraphy station in 1912). During the Cold War, GCHQ relied on an army of National Service personnel to support its monitoring activity. Conscripts, including the playwright Alan Bennett, were put through language courses and set to work listening to Soviet traffic. By the 1950s, large numbers of personnel went through the Joint Services School for Linguists, formed in 1951 to give language training, mostly in Russian, with a generation of national servicemen providing cheap labour and a pool of SIGINTers ready in case war started. It finished with the end of peacetime conscription in 1960.

Significant advantages

For Britain’s allies, its reach had significant advantages. In wartime, the global nature of Britain’s intelligence effort was reflected in the list of overseas intercept sites: Malta, Cairo, Sarafand in Palestine, Abbottabad and Delhi in India, Singapore and later Kilindini in East Africa and HMS Anderson, Ceylon (now Sri Lanka). Even as Britain entered a period of imperial decline, the ‘residual empire’ – the network of airbases, naval facilities and military sites from the Middle East to Singapore and Hong Kong – still mattered. For the Americans, Britain’s SIGINT offered Washington ‘an unrivalled network of … facilities that served US foreign policy interests’. Many were ‘indispensable’ for US national security. Hong Kong was one such jewel in the crown, offering secure facilities to monitor Communist China. Even during the Vietnam War, when London rejected formal requests to intervene, GCHQ provided the United States with vital support. Equally important were the listening posts in Cyprus, offering unrivalled coverage of the Middle East, while facilities in Africa were just as important as the Cold War turned to the ‘Third World’. In several cases, though having to give up formal control, Britain maintained SIGINT sites long after independence. Though satellites and imperial retrenchment ended this global reach, geography is still important, as the access to fibre optic cables, revealed by Edward Snowden, demonstrate.

‘Our objectives remain the same’, says GCHQ director Jeremy Fleming. ‘GCHQ has been at the heart of the nation’s security for a hundred years. It is a world-class intelligence and cyber agency full of fantastically committed people using their ingenuity to keep the United Kingdom safe.’ From ciphers to early computers, to mastering the internet to today’s cyber threat, GCHQ has also been at the forefront of UK technology, even if government monopoly on some of the latest tech is threatened. GCHQ’s history can be divided into the ‘first’ and ‘second’ ages of SIGINT. In the first, GCHQ and its predecessors emerged, becoming an established part of the British government. To those working for the organisation, the essential tasks remained similar: state-based threats largely using the same technology. Britain’s codebreakers also had the edge. Thanks to the work of the engineer ‘Tommy’ Flowers, Bletchley had Colossus, the world’s first programmable, electronic, digital computer, used against the messages of Germany’s high command. By the 1960s and 1970s, satellites and computers were changing the nature of SIGINT. Even if Britain often lagged far behind the US in the computer age, GCHQ remained a hub of innovation. It was one of the only UK government departments allowed to buy US-made IBM computers. Other departments had to put up with Britain’s home-grown products. GCHQ was also at the forefront of developing public key cryptography, a discovery only declassified in 1997. Such discoveries – and the academic discussions that resulted – formed the basis of the secure internet, enabling the day-to-day bank transactions and communications we take for granted.

A bewildering jungle

But the new ‘second age of SIGINT’, starting in the late 20th century, poses significant challenges. First, the target has changed. State-based monitoring is important, but the end of the Cold War and the changed nature of international relations saw non state-based threats – terrorism, serious organised crime – rise up the agenda. As the CIA Director James Woolsey remarked at the time: ‘We live now in a jungle filled with a bewildering variety of poisonous snakes.’ Targeting groups or individuals likely to conduct terrorist attacks has never been so important, while the proliferation of technology carries real risks. Whereas in the 1970s GCHQ was the first in the UK to get a Cray supercomputer, the complexity of modern communications means that most citizens have sophisticated technology at their fingertips. A surveillance target being tracked by GCHQ can now use WhatsApp, the dark web or even try to hide in plain sight to avoid surveillance. Moreover, while the Cold War monitoring of Soviet military networks was unlikely to get anyone killed, the nature of today’s terrorist threat means that one mistake can lead to a deadly attack. GCHQ now has to compete increasingly with the private sector for its staff and new technology. Apple, Microsoft and other tech giants are pushing the boundaries of online privacy, with difficulties for GCHQ and other western intelligence-security agencies.

But despite evolving communications and tech revolutions, GCHQ’s mission – and many of the problems it faces – remain much the same. Although Bletchley has attained a mythical-like status, Britain has a much longer and more varied history of codebreaking.

Daniel Lomas is Lecturer in International History at the University of Salford.