The idea that a meaningful agreement can be forced upon countries is farcical, writes Joseph Zammit-Lucia, we need co-operation on achievable policies
Predictably, COP19 in Warsaw has achieved little. Maybe the biggest achievement is that is has now become abundantly clear that the prospects are now close to zero that a meaningful legally binding, global agreement on carbon emissions will be signed in Paris in 2015.
Of course, some agreement may well be signed to enable all to claim success. But that can only happen if a form of words can be found to make such an agreement largely meaningless. As famously said by Geoffrey Howe, by drawing on the ability to separate words from meaning.
For many years, the COP negotiations have achieved little in spite of all the resources mobilised and the taxpayer money spent. Now, however, they have gone beyond being useless to being actually counter-productive to the cause of mitigating and adapting to climate change. Why?
Negotiations pitched in the framework of achieving legally binding emissions targets were always likely to result in confrontation by pitching one country's interests against the other. This generates antagonism and defensiveness rather than co-operation.
Many delegations no longer come to these talks armed with constructive suggestions. Rather, some focus on pushing others to make concessions in their own particular interests while others simply want to make sure that nothing gets written that could harm their national interests. The negative impact of this goes beyond the bland statements and paralysis to which we have now become accustomed. It is a framework that generates suspicion, mistrust, antagonism and casting of blame this way and that.
The endless and tiresome public condemnations emerging from environmental groups further increase the negative ambiance and governments' siege mentality. All this has been further escalated in Warsaw with ill-advised attempts to try to hold developed countries legally liable for the effects of past emissions. If there ever was an idea designed to irritate and turn un-cooperative countries that were previously helpful, this was a master stroke.
All this anger, mistrust and finger pointing came to a head in Warsaw when a whole gaggle of delegations simply walked out of the talks.
In such a poisonous atmosphere, no meaningful progress is possible. Worse than that, institutions such as the US Congress will, on principle, simply never ratify an agreement that they perceive as having been pushed upon them by other countries. China and Russia will likely take the same attitude. The idea that a meaningful agreement can somehow be forced upon the whole international community is farcical.
A primary issue is that the COP process is focused on the wrong target. There is very little disagreement on the reality of climate change. Neither is there much resistance to the idea that reducing emissions and preparing through adaptation are both desirable goals.
But no government is comfortable with what would be the appropriate policies to achieve these twin goals without compromising their economies or spending billions of non-existing taxpayer funds. It's not a lack of will that is the problem; it's a lack of politically and practically achievable ways to achieve these results. In this environment, it is sheer folly to attempt to impose targets without a meaningful road map of how such targets can be practically achieved.
What would be a productive way forward? The first, overwhelming need is not for a legally binding agreement but for an atmosphere of positive co-operation. We can achieve this - over time - by converting these UN-sponsored meetings into conferences of achievement. Government delegations should be invited to share with others what they have done right, how they have done it, what they have learned, and what they are planning to achieve further progress.
Other governments can take whatever they want from these experiences and adapt them to their own circumstances. Small groups of countries whose interests may coincide can, if they wish, form formal alliances to work together to achieve progress. Larger, wealthier countries can offer help and support in technology transfer and financial assistance on a bilateral basis.
These conferences can become deal-making conferences where everyone can take what they wish in an atmosphere that is focused on positive achievement, where nobody is called out as the culprit, and where no attempts are made to hold individual countries' feet to the fire.
Second, the focus needs to change from emissions targets to devising appropriate and credible policies. In spite of the continued assertions of activists to the contrary, it has been shown time and time again that we currently have neither the technologies nor the social structures nor the policy know-how to convert developed economies quickly to low carbon ones without considerable economic and social cost.
As has been shown in Canada and Australia, over-reach simply results in electoral loss and quick policy U-turns. Neither do developing countries yet have the wherewithal to accommodate their development aspirations using a low carbon path. These meetings would be more productive if they were to focus seriously on the 'how' of low carbon economies rather than clamouring for targets that countries will never sign up to simply because they have no idea how they might credibly achieve them.
Finally, these meetings need to be taken out of the hands of scientists, technocrats and activists and handed over to institutions that understand the language, the practicalities and the difficulties of multi-lateral diplomacy. Countries should not be represented by environment ministries that tend to be dominated by technocrats and, as we have seen in Warsaw, simply get axed when they concede too much.
Ministries of foreign affairs, ministries of industry and the economy and overseas development agencies would provide a much more rounded and appropriate skills base. The voice of environmental organisations would be more effective and less disruptive if it were represented by a single, respected agent such as the International Union for Conservation of Nature – a membership organisations that counts all the major environmental NGOs, as well as many government organisations, among its members. A single voice providing constructive input is more effective than an un-coordinated free for all.
The continued march of climate change and its human, natural and economic impacts is a tragedy. The COP process has converted it into a farce. If we are to make progress, we need to dismantle the current poisonous atmosphere of conflict and mistrust and replace it with an atmosphere of co-operation. This can only be achieved by having the right people at the table talking about positive achievement and changing the focus from meaningless targets to exploration of practical and achievable policy innovations.
Join the community of sustainability professionals and experts. Become a GSB member to get more stories like this direct to your inbox
A new study examines meteorologists, the global warming consensus, political ideology, and climate expertise
Several surveys have found relatively low acceptance of human-caused global warming amongst meteorologists. For example, a 2009 survey found that among Earth scientists, only economic geologists (47 percent) had lower acceptance of human-caused global warming than meteorologists (64 percent). A new paper by social scientists from George Mason University, the American Meteorological Society (AMS), and Yale University reports results from a survey of members of the AMS to determine the factors associated with their views on climate change.Climate Scientists and Meteorologists, Apples and Oranges
Predictably, many climate contrarians have already misrepresented this paper. In fact, the Heartland Institute (of Unabomber billboard infamy) misrepresented the study so badly (and arguably impersonated the AMS in a mass emailing), the AMS executive director (who is a co-author of the paper) took the unusual step of issuing a public reprimand against their behavior.
The misrepresentations of the study have claimed that it contradicts the 97 percent expert consensus on human-caused global warming. The prior studies that have found this high level of consensus were based specifically on climate experts – namely asking what those who do climate science research think, or what their peer-reviewed papers say about the causes of global warming.
The AMS on the other hand is not comprised primarily of climate experts. Some of its members do climate research, but only 13 percent of survey participants described climate as their field of expertise. Among those respondents with climate expertise who have published their climate research, this survey found that 93 percent agreed that humans have contributed significantly to global warming over the past 150 years (78 percent said it's mostly human-caused, 10 percent said it's equally caused by humans and natural processes, and 5 percent said the precise degree of human causation is unclear, but that humans have contributed). Just 2 percent of AMS climate experts said global warming is mostly natural, 1 percent said global warming isn't happening, and the remaining 4 percent were unsure about global warming or human causation.
The authors also note that they asked about contributions to global warming over the past 150 years, whereas climate scientists are most confident that humans are the dominant cause of global warming over the past 50 years. Some survey participants sent emails implying that if the question had more narrowly focused on the past 50 years, even more respondents might have said that global warming is mostly human-caused.
Importantly, most AMS members are not climate researchers, nor is scientific research of any kind their primary occupation (for example, weather forecasters). Among those AMS members who haven't recently published in the peer-reviewed literature, just 62 percent agreed that humans are causing global warming, with 37 percent saying humans are the main cause over the past 150 years.
The bottom line is that the previous studies finding 97 percent expert consensus on human-caused global warming are not directly comparable to this new study, which surveyed all AMS members, most of whom are not climate experts. The study's lead author, Neil Stenhouse, agrees that the Heartland Institute's James Taylor has misrepresented their study.
"Mr. Taylor's claims are highly misleading, but we expect that from someone with a long history of distorting the truth about global warming. We found high levels of consensus on global warming among the climate experts in our sample. You only see low levels of consensus in the sample if you also look at the views of people who are not climate experts."What's Causing Meteorologist Skepticism?
When we actually examine the questions the study does investigate, as opposed to the contrarian approach of twisting the results to try and make them fit preconceived notions, it contains a lot of interesting information. The authors proposed four hypotheses to explain the variation in AMS members' views on global warming. They found evidence supporting each of the four hypotheses. In terms of predicting meteorologists' positions on human-caused global warming, listed in order from strongest to weakest, these were:
1) Perceived scientific consensus on global warming
2) Political ideology
3) Climate expertise
4) Perceived conflict among AMS members on global warming
Interestingly, the strongest single factor in predicting meteorologists' acceptance of human-caused global warming was their perception of the level of expert consensus on the subject. This result is consistent with previous research finding that people are more likely to accept this reality and support taking climate action if they're aware of the expert climate consensus. Like most people who are not expert in a particular field, most meteorologists also defer to the expert consensus...when they're aware that expert consensus exists. This is precisely why climate contrarians work so hard to deny that the climate consensus is real. The authors suggest tackling this misconception head-on.
"First, the strong relationship between perceived scientific consensus and other views on climate change suggests that communication centered on the high level of scientific consensus may be effective in encouraging engagement by scientific professionals."
Political ideology was the second strongest predictor of meteorologists' positions on global warming. Conservative AMS members were significantly more likely to doubt the reality of human-caused climate change. This tells us that the relatively high rate of rejections of the climate consensus isn't based on science, because the scientific evidence has nothing to do with politics. This is also evident from the fact that meteorologists with more climate expertise are more likely to accept human-caused global warming. According to this study, the relatively low level of consensus across all AMS members is due to a combination of factors: lack of awareness of the expert consensus, political bias, and lack of climate expertise.
In any case, the 97 percent expert consensus on human-caused global warming is still a reality.Dana Nuccitelli
There are some common misunderstanding among scientists about how governments make their policy decisions
Top 20 things politicians need to know about science
When scientists moan about how little politicians know about science, I usually get annoyed. Such grouching is almost always counterproductive and more often than not betrays how little scientists know about the UK's governance structures, processes, culture and history.
So when the Guardian reported on a Nature article that listed 20 things that politicians should know about science, I started reading it with apprehension, half expecting my head to explode within a few paragraphs.
I needn't have worried. The authors, Professors William Sutherland, David Spiegelhalter and Mark Burgman, have produced a list that picks up on many of the challenges that scientists report when engaging with policy makers, and it does so in a constructive way.
It has been printed out and stuck on my wall. It has been passed around the office. I am sure to reference it often.
But the fact remains that all too often, scientists blame politicians for failures when science meets policy-making, when in truth the science community needs to do much more to engage productively with the people who actually make policy.
There are similarities with long-standing and successful efforts to improve the relationship between science and the media. Fiona Fox, the director of the Science Media Centre, has repeatedly and correctly asserted that "the media will 'do' science better when scientists 'do' the media better". I believe that the same is true of science-policy connections.
So here is a list of 20 things that I and my fellow science advisers at the Parliamentary Office of Science and Technology think scientists should know about policy. We knocked up the list in just an afternoon last week but it should stimulate debate, and if anyone were to print it out and stick it on their wall or pass it round their labs, it probably wouldn't do any harm.
Here we go, in no particular order …1. Making policy is really difficult
The most common science rant about policy making includes a flippant comment about policy decisions being straightforward. I've heard people say that it is "obvious" that the UK should decriminalise drugs; stimulate the economy by doubling the science budget; reform our energy economy by investing extensively in nuclear. Such decisions are not straightforward at all. Public policy is always more complex than it seems, involving a wide range of inputs, complicated interactions with other policies, and varied and unpredictable outcomes. Simple solutions to complex problems are rarer than most people think.2. No policy will ever be perfect
Whatever the decision, the effects of policy are almost always uneven. For example, any changes to taxes and benefits will leave some people better off and other worse off; and while the research impact agenda has been undoubtedly positive in some respects, it has caused problems in others.3. Policy makers can be expert too
Scientists often consider themselves as the "experts" who engage with policy makers. In my experience, many policy makers are experts too. Some have excellent research credentials, and frequently they understand the research base well. When I worked at the University of Cambridge, one of my jobs was to connect researchers to policy makers; the researchers often told me how much they learned from speaking to policy makers. In other words, if you are a scientist talking to a policy maker, don't assume that you are the only expert in the room.4. Policy makers are not a homogenous group
"Policy maker" is at least as broad a term as "researcher". It includes civil servants ranging from senior to junior, generalist to specialist, and to those in connected agencies and regional government; it includes politicians in government and opposition, in the Commons and the Lords; and then there are all the people who might not directly make the decisions, but as advisers can strongly influence them.5. Policy makers are people too
See number 12 of the Sutherland, Spiegelhalter and Burgman list. Policy makers are people who, despite extensive training and the best of intentions, will sometimes make bad decisions and get things wrong. Also, they may – like scientists – choose to act in their own interest …6. Policy decisions are subject to extensive scrutiny
… which is why, like science – which mitigates human nature insofar as it is possible with the principles of academic rigour and peer review – policy is regulated by professional guidelines, a variety of checks and balances, and scrutiny that comes from a wide range of institutions and angles. For example, Parliament scrutinises government and the House of Lords scrutinises the House of Commons.7. Starting policies from scratch is very rarely an option
A former government minister once told me that, on taking office, he decided to meet with a number of academics to seek advice on how to fix his particular policy domain – which was, and still is, largely broken. He found the experience to be deeply frustrating because everyone he met said: well, if you were designing the system from scratch, this is what it should look like. But he wasn't; he needed solutions that could evolve from within the existing ecosystem. This rule applies in a lot of policy areas, from infrastructure to education, from the NHS to pensions.8. There is more to policy than scientific evidence
Policies are not made in isolation. First there is a starting point in current policy, and there are usually some complex interactions between policies at different regional scales: local, national and international. This is true of policy areas such as drugs, defence, immigration and banking regulations. Law, economics, politics and public opinion are all important factors; scientific evidence is only part of the picture that a policy maker has to consider. Most of the major policy areas that consistently draw opprobrium from scientists are far more complicated than just scientific evidence: energy, drugs and health, to name just three.9. Economics and law are top dogs in policy advice
When it comes to advice sought by policy makers, economics and law are top dogs. Scientific evidence comes further down the pecking order. Whether or not this is the best way to make policy is not the point, it is just a statement of how things work in practice.10. Public opinion matters
Many of the most important public policy decisions are made by people who were directly elected, and most of the rest are taken by people who work for them. We live in a democracy and public opinion is a critical component of the policy process. The public is directly involved in many planning decisions and public opinion is a consideration in the distribution of healthcare providers, schools and transport services. Complex policy areas such as drugs, alcohol, immigration and education, are all heavily influenced by public opinion.11. Policy makers do understand uncertainty
It is commonly asserted by scientists that policy makers prefer to be given information that is certain, and I have even heard some say that policy makers don't understand uncertainty. On the contrary: politicians are surrounded by and constantly make formal and informal assessments of uncertainty (for example, when considering polling information) and civil servants are expert at drawing up policy options with incomplete information (which is just as well because complete information is a fantasy). It is true to say that policy makers are not fond of information so laden with caveats that it is useless. Better than hazy comments about policy makers not understanding uncertainty, the Sutherland, Spiegelhalter and Burgman list is a productive explanation of what knowledge and skills would help policy makers.12. Parliament and government are different
In the UK, the distinction between parliament and government is profound. Parliament – the legislature – debates public issues, makes laws and scrutinises government. Government – the executive – is led by select members of parliament and is responsible for designing and implementing policy. Parliament is made up of over a thousand MPs and peers, with a small staff of only a few thousand. Government is made up of only a hundred MPs and peers, with a staff of hundreds of thousands. For the record, I work in parliament.13. Policy and politics are not the same thing
Policy is mostly about the design and implementation of a particular intervention. Politics is about how the decision was made. Policy is mostly determined in government, where the politics is focused by ministers, the cabinet, and the party leadership. In the House of Commons, there is less policy and more politics.14. The UK has a brilliant science advisory system
The UK is leading the world with its science advisory system. Every government department has (in theory) a chief scientific adviser reporting to his or her own private secretary (the top departmental civil servant) and to the government chief scientific adviser, who reports directly to the prime minister. In parliament, we have the Parliamentary Office of Science and Technology, and in addition science advisers in the library research services and select committee offices.15. Policy and science operate on different timescales
When policy makers say that they need information soon, they mean within days or weeks, not months. This is not a flaw of the system; it is the way it is. If scientists want to engage with policy they need to be able to work to policy makers' schedule. Asking policy makers to work to a slower timetable will result in them going elsewhere for advice. And make your advice concise.16. There is no such thing as a policy cycle
I have seen many flow charts depicting "the policy cycle". They usually start with an idea, move through a sequence of research, design, implementation and evaluation, which then feeds back into the start of the cycle. Fine in theory, but in practice it is a lot more complicated. Policy making is iterative; the art of the possible.17. The art of making policy is a developing science
We live in exciting times for policy making. Various initiatives for better governance are under way, including ones for opening up the policy making process, and others for building evaluation into policy implementation. The new What Works Centres are roughly based on Nice (the National Institute for Health and Care Excellence, the healthcare body that recommends which treatments the NHS should use), but instead it will consider how to reduce crime, stimulate local economic growth, promote better ageing and use early intervention better. Research evidence, particularly from the social sciences, will play a key role. In another innovation, the Cabinet Office is set to establish a Policy Lab.18. 'Science policy' isn't a thing
When policy makers talk about "science policy", they are usually talking about policies for things like research funding, universities and innovation policy. Researchers additionally use "science policy" to talk about the use of research evidence to help deliver better policies in a wide range of areas. I find it helps to distinguish between "policy for science" on the one hand, and "science for policy" on the other.19. Policy makers aren't interested in science per se
Well some are, but on the whole, policy for science is pretty niche. Policy makers tend to be more interested in research evidence to inform policy making, but let's be clear: they are not interested in philosophical conversations such as "what constitutes evidence" or "the difference between science advice, social science advice and engineering advice". Policy makers care about research evidence insofar as it helps them to make better decisions.20. 'We need more research' is the wrong answer
Policy decisions usually need to be made pretty quickly, and asking for more time and money to conduct research is unlikely to go down well. Policy makers have to make decisions with incomplete information (see #11) so they may exhibit frustration with researchers who are unable to offer an opinion without first obtaining funding for a multi-year research programme. I'm not saying that more research isn't often needed; it is. But it is not the answer I would ever choose to give to a policy maker seeking scientific advice.Chris Tyler
The TV illusionist wants to make a point about our invisibility by training four of us to steal a painting – but we are perfectly capable of behaving badly without any help thanks very much
Derren Brown, illusionist, has realised that his parents are growing older – well done Derren – and that old people have "richer stories to tell". Now aware of our potential, he's going to train four of us to steal a painting from a London gallery. He could have chosen youngsters, but he wants to help elderly people by "making a point" about our "invisibility". It's not a bad idea, even though he has dredged up the same dreary old stereotype. Four drab old farts creeping round a gallery probably won't look suspicious. Yet we're quite capable of flouting the law and dressing like peacocks without help.
My friend Olga did it last week. Her hair is striped, her clothes a riot of colour, Camden Town was jammed, as usual, and Olga was stuck on a bus, moments from her stop. Ten minutes passed, passengers started grumbling, Olga asked the driver to let her out, but he wouldn't open the doors.
"It's more than my job's worth," said he. "You can get out yourself if you want." But he wouldn't tell her where the door-opening button was, so, as she's a potter, used to heaving clay about, she and a young man wrenched the doors open with their bare hands and Olga escaped. The young man didn't dare, because it wasn't allowed. How wet.
Rosemary also escaped a bus, but without wrenching, because the kind conductor showed her the button above the door. She often bravely challenges loutish behaviour – gobbing, littering and suchlike – and so did my mother, once calling a huge, muscular, bare-chested fellow an "arsehole", because he deserved it, and she was fearless. What did she care? She was keen to peg out anyway. Perhaps this is why some of the elderly are bold as brass, because they no longer give a toss about others' opinions. We're free at last, to go wild or turn to crime. Thirty-three over-70s were arrested in Kent last year, a hotbed of elderly criminals. We are facinating and can manage ourselves, but thanks, Derren, for giving me the chance to point this out. Again.Michele Hanson
Materials scientist and BBC Science Club presenter Mark Miodownik attempts to show that electronics can be as enjoyable and creative to play with as paints and clayPaul BoydIan Anderson
A stunning time-lapse HD video of an artist at work drawing a beautiful, realistic parrot
Sometimes, watching an artist at work is almost like watching magic happen. A flat surface of paper or canvas is transformed into a three-dimensional picture that looks real. In this case, I was so impressed by Italian artist Marcello Barenghi's drawing of a blue-and-yellow macaw, Ara ararauna, that I had to share the stunning time-lapse video of its creation with you.
The blue-and-yellow macaw -- also known as the blue-and-gold macaw -- is a large parrot with a long pointed tail. These parrots are sapphire blue with lemon yellow underparts, an emerald green forehead, a charcoal black throat and a white face. Blue-and-yellow macaws are large; adults measure between 76 to 86 cm from beak to tail-tip, and weigh between 900 to 1500 grams (2-3.3 pounds).
These parrots live throughout dry woodlands of South America in family groups and loose flocks. Although the blue-and-yellow macaw is nearly extirpated from some areas (Paraguay in particular), it nevertheless is common throughout much of its range, so it is classified as "least concern". Wild blue-and-yellow macaws are protected from capture, trade and hunting under international law.
Wild blue-and-yellow macaws consume a tremendously varied diet of fruits, vegetable material and nuts. Their large beak evolved as a multi-purpose tool that the parrots use to break open the hard shells of nuts, to redecorate nest cavities and they also use it as a "third hand" for climbing.
Although loud and destructive, captive-bred blue-and-yellow macaws are commonly kept in the United States and throughout parts of Europe because, when handled respectfully and often, they make gentle and affectionate pets. They are long-lived, intelligent and trainable and they require a lot of attention and interaction on a daily basis. These parrots can mimic the human voice, although I think they sound like they are shouting rather than chatting.
As promised, here's a time-lapse video of the artist creating a drawing of this parrot species. I particularly enjoy that special moment when the drawing goes from being a flat representation to something more -- when the parrot seems to pop right off the surface. This 3 minutes and 48 seconds long video captures the entire creation process, which the artist says took 3 hours and 52 min to complete:
Illustrator Marcello Barenghi specialises in creating hyperrealistic speed drawings for HD videos. He maintains an extensive website filled with time-lapse videos of him drawing a huge variety of subjects.GrrlScientist
In 2011 Ferran Adrià closed his globally famous restaurant. He has turned it into a food research institute which includes Bullipedia, a project to map all foods and their ingredients
Professor Adrian Cheok was feeling a bit nervous. In October, he and his small team of postgraduate students learned that their invention was one of four finalists in the HackingBullipedia contest, and although the accomplished professor at City University in London, where he is also director of its Mixed Reality Lab, has received a slew of awards for his work on the interface between humans and computers, this contest promised to be different. For one thing, although there would be plenty of IT and computing experts on the jury, the most important judge was a chef. And not just any chef, but Ferran Adrià.
Cheok had reason to be apprehensive: Adrià is widely considered the most influential chef of our time. At his restaurant elBulli, located on the north-eastern coast of Spain, he revolutionised cuisine by bringing a radical, scientifically inflected creativity to bear on the process of cooking. The techniques and ingredients he developed in his kitchen and in his famous lab just off of Barcelona's Boqueria market – the foams, the sands, the liquids turned into tremulous globules via the magic of spherification – produced some startling dishes: "dragon" cocktails that made the drinker breath smoke; "caviar" made from pearls of olive oil; a robust plate of lapin royale made into gelatin. Hot into cold, sweet into savoury, solid into liquid or air, during the 27 years he ran the restaurant, Adrià played with his diners' expectations, undermined established categories of taste and texture, and constantly, miraculously, continued to surprise. "Presenting our prototype to him," explains Cheok, "is like performing a song for Michael Jackson."
Luckily for Cheok, Adrià won't be judging the team for its cooking skills, but for its imagination. Indeed, ever since elBulli shut its doors to the public in 2011, the chef has been busy turning the restaurant into the elBulli Foundation, the primary mission of which is to foster not just good cooking, but greater innovation in the kitchen and out of it. "I've been very successful, and very lucky, and now I want to give something back," he says. "And the best thing I can give is teaching others what I know, which is creativity."
Creativity was always at the core of what Adrià did as a chef. By marrying scientific techniques and processes to his wild imagination, he developed what he called "a new language" of cuisine. He and his cooks learned to use sodium alginate and calcium carbonate to turn liquids into "spherifications"; to employ liquid nitrogen to create a frozen parmesan foam; to add hydrocolloids to thicken sauces without flour or butter.
Few of these innovations were actually his inventions. Most of the novel techniques he employed had been used previously in industrial food production but had never before been applied to fine dining. And despite the sometimes mocking depictions of him as a test-tube wielding scientist, Adrià refused to distinguish his techniques as essentially distinct from anything chefs had always done. How was a PacoJet any more alien a machine than an oven? Why did we consider maltodextrose a processed "additive" and sugar a "natural" ingredient? Once when, in the course of one interview, a journalist enquired yet again about the gadgetry and chemistry-set cooking he had heard so much about, a mildly exasperated Adrià pulled the hapless reporter into the kitchen. "You want to see my technology?" he demanded. He grabbed the hand of one of the line of apprentice chefs busy shelling pistachios, and turned it over to revealing red fingers sore from hours of meticulous labour. "There's my technology."
In other words, Adrià rejected the notion that he was somehow breaking with what normal chefs did, just as he rejected the idea that his style of cooking represented a rupture with the centuries of cuisine that had preceded it. His style of cooking – often called "molecular gastronomy" although Adrià reviled the term – was an evolution, just as nouvelle cuisine evolved from the traditional cooking of France. Now, as he moves onto the next phase of his career, the ways in which he is bringing technology from other disciplines to bear on cuisine are evolving again.
"ElBulli never closed," Adrià says, "We just stopped serving meals." Indeed, ever since the chef stunned the food world in January 2010 by announcing he would no longer be serving food, he and his team, which involves a partnership with the Madrid-based telecommunications giant Telefónica, have been hard at work designing the elBulli Foundation. At first, its shape and purpose were noticeably vague ("The only thing I can tell you for sure," said Adrià at one point, "is that it won't be a cooking school. We won't be teaching anyone to break down a cod.") Three-and-a-half years later, however, several concrete projects are underway.
Perhaps the easiest to understand is elBulli 1846. It wouldn't be an exaggeration to call Adrià and his crew hoarders, albeit very, very organised ones. Over the years, they have kept track of just about everything, from the prototypes for spoons designed to serve the restaurant's famous spherified olives to Adrià's tasting notes on a black-and-white sesame dish called ying-yang, with a slew of magazine covers and cartoonist Matt Groening's depiction of Ferran in full The Simpsons style. All of those items will be on display in a new museum and archive that will be housed in Roses, the town on the Catalan coast nearest to elBulli (the 1846 refers both to the number of recipes created at elBulli in Adrià's time there, and to the year in which the famous French chef Auguste Escoffier was born.) Adrià says he expects 200,000 visitors a year to the museum.
Eight kilometres up a winding coastal road to the village of Cala Montjoi, the former restaurant will become a research centre, named elBulli DNA, and will form the second of the foundation's key projects. There, the old tchotchke-filled dining room and sleekly modern kitchen will be preserved intact, but will be complemented by new buildings with vaguely organic shapes that recall both Gaudí and Alice in Wonderland, to be designed by the Catalan architect Enric Ruiz-Geli. The complex will include administrative offices, a screening room and archives, but its beating heart will be the old elBulli kitchen, which will be transformed into a hive of culinary experimentation of the sort that Adrià and his team used to conduct at their workshop in Barcelona during the six months that the restaurant was closed each year. But instead of inventing new dishes or techniques for the restaurant's lucky diners, the cooks will be sharing their findings with the world.
Thanks to support both financial and technical from Telefónica, the elBulli DNA test kitchen promises to be among the most advanced in the world. But with Adrià's new emphasis on spreading ideas instead of just fomenting them, it will also be tailored to promote different kinds of interaction. Each day, the fellows at elBulli DNA will upload the results of their work so the rest of the world may follow. Sensors will be used to track both the people and the tools used to prepare a dish, to enable those trying to recreate it in a kitchen in Singapore or London to do so with greater precision. Screens within the kitchen will make it possible for a cook in the cold station to see what another is doing at the stove. And interfaces that rely on gesture rather than touch will enable cooks whose hands are otherwise occupied to access the online database. Other innovations, says Maria José Tomé, innovation manager at Telefónica R&D, are sure to come. "Ferran sees things in a different way," says Tomé. "We are working with him in part because we want to support innovation, and in part because we see this as a way of generating new opportunities for the company, new products that accelerate the creative process and that we may be able to commercialise."
One of elBulli DNA's first projects, already underway, is to develop a comprehensive classification – a taxonomy, of sorts – of all food products: their biological origins, their relation to other foods, even how they got their names. "You can't fully create if you don't understand what you're working with," Adrià says. "What's the difference between ravioli and empanadas? They're both dough stuffed with something. How can I come up with a new take on them if I can't explain what each one is?" At a gathering with chefs and seed breeders in New York in September, he delighted in approaching the scientists and asking each to explain the difference between a fruit and a vegetable. "You see?" he would cackle as one after another stumbled over the answer. "We don't know anything. I'm Ferran Adrià, I've been cooking for 30 years, and even I don't know the answer to something like that."
The taxonomy will be created in-house, and turned into software that culinary schools and other institutions can use. Those characteristics distinguish it from the foundation's third great initiative, named Bullipedia. The goal of that database, however, is no less daunting. In the modest language of its creators, it will "organise in a clear, ordered and precise way all culinary knowledge". Luckily, Adrià and his team will have some help: the database will be created by the same professionals for whom it is intended. Culinary schools, restaurants, and universities will all contribute.
Although the comparisons –including in the name – to Wikipedia are unavoidable, the Bullipedia will not be based on crowdsourcing per se, but rather on the work of a select, vetted group of contributors. And because it is important to the foundation that all the information in the Bullipedia be absolutely accurate (another difference from Wikipedia), the foundation will provide the documentation and templates that contributors need to craft their entries.
What that means in practice remains to be seen. But planners – including researchers at the University of Barcelona, which will oversee the Bullipedia – are working with the idea that a user searching within the database for information on, say, broccoli, would come up with entries that included not only recipes for dishes simple and elaborate, but also biological information on brassicas, suggestions for flavour pairings, information on optimum growing and harvesting conditions, techniques for cooking and preserving it, and notes about where the next steps in broccoli innovation may lie.
Does the world – even the rarified world of chefs – really need so baroque a compendium of knowledge? But Adrià's motivation for compiling and organising goes far beyond the mere knowledge itself. "Order, order, order, that's how you create," he says. And indeed, both the genome and the Bullipedia can be seen as continuations, albeit on a much grander, more comprehensive scale, of the archival work that Adrià and his team used to do back when they were actually feeding people. From early on, the restaurant meticulously catalogued its work, not just the successful recipes and techniques that made it into the dining room, but the hundreds of failures along the way. That tedious labour may not fit most people's romanticised vision of inspiration, but it was critical to Adrià's creative process. If he got a notion to come up a new dish featuring pine nuts, for example, he could go back through years of notebooks to find everything he had ever done to them – cooked them in a creamy broth to make a pine nut risotto; infused them in pinecones filled with boiling water for a pine nut tea – and build from there.
The Bullipedia takes that basic idea, that from order comes the anarchy of creativity, and multiplies it a thousand-fold. By taking what was originally a purely written form of documentation – later, Adrià's crew would also take photos of finished dishes – and making it digital, the information contained in the Bullipedia will be available to anyone with an internet connection. By making the compilation of knowledge interactive, it guarantees that the Bullipedia will remain a living document, able to change and react to any new invention or discovery.
To jump-start the collaboration, Adrià and his partners from Telefónica launched the Hacking Bullipedia contest. The competition drew dozens of entries. Cheok's submission was one of the four finalists It is for a device that uses electric stimulation of the taste buds, coupled with a chemically produced scent emitted from a device that attaches to a mobile phone, to reproduce the flavours of any food or dish. "This is a new kind of virtual reality that allows users not just to see food on the internet but to taste and smell it," he says. Cheok imagines that the invention could be useful to a chef who, in looking through the Bullipedia, wanted to know what the spherified olives she read about there tasted like, or who wants to use the knowledge contained within the work to create new flavours altogether.
Is all this folly? With anyone else, it would be easy to say yes. But Adrià has been called crazy before: when he decided to close six months of the year; when he opened a workshop dedicated solely to experimentation. In each of those cases, and in many others, he triumphed. And throughout, Adrià has been consistent in his evolution. In fact, all of the changes described above were undertaken in the quest to keep alive his remarkable ability to innovate.
Back in January 2010, when the Catalan chef announced that he was closing elBulli, the idea that so successful a restaurant would voluntarily close was unthinkable to many, and foodie circles speculated that an ugly reason – health problems, money issues, a brewing lawsuit – lay beneath the unexpected decision. But in a world in which boundless ambition is taken as an obvious good, the truth was even more scandalous: Adrià closed the restaurant in order to keep doing what he had always done best, which was not to make money or be successful or even to cook, but rather to innovate.
Three years later, as the elBulli Foundation's shape coheres, Adrià is more enthusiastic about the project than ever. "I'm no longer interested in creating dishes," he says. "I'm interesting in creating the creators of dishes."
Revelations show transatlantic intelligence pact started in second world war is expanding beyond states' ability to control it
There haven't been too many moments of levity over the past four months for those intimately involved in the story of Edward Snowden. It hasn't been a laughing matter for the man himself, who is now stuck in Russia, the intelligence agencies whose secrets he has disclosed, or the governments that have had to deal with the consequences.
But the impasse between the opposing forces in this unprecedented and complex saga has been broken on occasion. One of these moments came at the Guardian's London headquarters, near King's Cross station, on Wednesday 17 July.
The scene was a second-floor office overlooking Regent's Canal, the time 11am. On one side of a large, round wooden table sat two senior officials from the Cabinet Office, nursing cups of coffee and unconcealed irritation. Facing them were two journalists from the Guardian.
After hollow pleasantries and firm handshakes, the conversation turned to the right to freedom of speech on issues that might affect national security. And, though no voices were raised, the message – which had come directly from the prime minister – was loud, clear and intended to unnerve.
The Guardian had become a target for every intelligence service in the world, intoned the grey-suited official. His colleague nodded. She took notes. Hostile foreign agencies would be using all manner of low tricks and high technology to get hold of the classified files gifted to us by Snowden.
The tactics might include anything from pointing long-range lasers at plastic cups used by our reporters (very good for eavesdropping apparently), to bribing members of our staff. Had we recruited anyone in the past few weeks, they inquired? Any Chinese, perhaps?
The Guardian had to be alert to such dangers because the threat could come from anyone, anywhere, at any time.
"Some of the best assets of the best intelligence services in the world will be interested in you," said the bespectacled official. At precisely that moment, and with implausibly good timing, two window cleaners slowly dropped into view on the outside of the building. In a cradle hanging from the roof, they soaped, swiped and polished, moving slowly up and down just a few feet away.
"Are they yours or the Chinese?" said one of the editors. Even the mandarins managed a smile.
For a few moments the absurdity of the situation overwhelmed the seriousness; and if it wasn't quite football between the trenches at Christmas, the meeting ended cordially, with both sides recognising the difficulties of the other.
Since then, the debate has become rather more polarised, entrenching views at a time when a more rounded and less doctrinal discussion might be better for the people who really matter in all this.
Those people do not include Snowden or the reporters working on the stories; or the directors of intelligence who have been so affronted by the disclosures; or even the presidents and prime ministers on whose watches surveillance has entered a new, remarkable, era. The principal characters in this drama are not the giant computers used for storage, analysis and codebreaking, or the technicians who built them.
The people who really count are the millions who send emails or search on Google or use mobile phones – and expect privacy. Those who use Skype, send direct messages on Twitter, post on Facebook or rely on the internet to buy groceries also make up the cast. Because all this information, whatever safeguards you have taken, can be swept up, decoded and analysed by British and American intelligence agencies.
Arguments over Snowden's motives, whether he is a whistleblower or a traitor, whether his disclosures have damaged the agencies or just embarrassed them, may never be reconciled. But on one matter there is no doubt. The highly classified files have shown that espionage has changed.
The world immortalised by John le Carré drew a distinction between those who were in "the business" and those who were not. This withered with satellite communications and died when everyone began to research and speak to each other online. We are all part of "the business" now.
Without offering details to anyone outside their inner circles, western intelligence agencies embarked on a new strategy – data trawling.
When the first of the Snowden revelations was published in the Guardian in June – revealing that the NSA was secretly storing and analysing details of millions of phone calls made in the US – the transformation was recognised immediately.
"The administration is saying that without any individual suspicion of wrongdoing, the government is allowed to know whom Americans are calling every time they make a phone call, for how long they talk and where," the New York Times said in an editorial.
"Through a series of legal contortions Obama has argued that Congress, since 9/11, intended to implicitly authorise mass surveillance. But this strategy mostly consists of wordplay, fearmongering and a highly selective reading of the law."
As the focus of the stories turned to the UK over the following weeks, it would soon become clear that all those criticisms could be levelled at Britain too. The UK hasn't just been a partner in this technological adventure, it has been a pioneer, with the two countries working more closely in the field of intelligence-gathering than in perhaps any other since the second world war.
The bombe and the Purple machine
The gardens at Woking crematorium in Surrey are neat and peaceful, and full of well-tended rows of red, pink and yellow autumn flowers. Under the blooms sit further rows of small, clean white postcards, on which mourners have written their thoughts and prayers for the deceased.
The crematorium was founded in 1878 (to the indignation of residents who didn't want the town to become the first in the country to have such a godless facility) and sits directly opposite the Winston Churchill sports centre.
A few of the dead have memorials here; many more have their names recorded in the books of remembrance that are stored in the vaults. Alan Mathison Turing is not one of them.
Turing was cremated here on Saturday 12 June 1954, five days after he died. His ashes were spread by his elder brother John, in Tennyson Lake Garden North, a secluded 1.6-hectare (four-acre) garden next to a pond where their father's remains had also been scattered.
Archives at the crematorium only record that Turing was 41 and that his occupation was "university reader". There is nothing else to mark his death.
The Times obituary, which appeared on the same day, noted Turing was a mathematician and logician who had branched into "the design and use of automatic computing machines".
The piece bemoaned how the second world war had "interrupted Turing's mathematical career for six critical years between the age of 27 and 33" – and his death had deprived the world of a man who could have "made much greater discoveries". Few people knew the truth: Turing was a master codebreaker, and during those "lost" war years, he had made one of the most important discoveries in British military history.
He had enabled the codes used by the Nazis to send messages to and from their commanders to be cracked.
The story of Turing and the team at Bletchley Park was not one GCHQ wanted to boast about in the 1950s or in the decades thereafter; his work was top secret, his private life complex and, for the time, scandalous.
Turing was gay. He killed himself by eating an apple laced with cyanide two years after he had been convicted of having a sexual relationship with a young man from Manchester.
Just about everything Turing did in his professional and personal life, and the environment in which he did it, has changed since his death.
But if you are tracing the roots of the relationship between GCHQ and the NSA, to understand why the agencies work so closely together, and why they seem so genuinely perplexed (and angry) by the furore now surrounding them, then it is to Turing and his contemporaries that you have to turn.
Looking back is also the only way to appreciate how the intelligence agencies have ended up on the path to mass surveillance, and managed to travel a long way down it, without facing the kind of public scrutiny they are confronted with now. The fathers of the institutions that have become security behemoths were men such as Turing and Wolf Friedman, an American cryptologist who was as brilliant as his British counterpart although somewhat less eccentric.
There had been codebreakers before these two, but their work was on the verge of a technological revolution that is still going on today.
Working in Hut 8 at Bletchley Park, then the home of GCHQ's forerunner, the Government Code and Cypher School, Turing found a way of reading messages sent by the Germans, using a codebreaking machine called the bombe. Across the Atlantic, Friedman developed a way of cracking the Purple machine, the device used by Japan to code wartime messages.
British and American spies had worked closely during the first world war, but the bonds were pulled tighter in the spring of 1941, when four US officials travelled to Bletchley Park to deliver a model of the Purple machine.
They received intelligence gifts in return and the exchange of information has been going on ever since.
During the second world war, codebreaking was the endeavour of small teams of gifted individuals, working with crude machines that hummed and whirred as cogs and wheels of varying sizes turned at different speeds.
The enemy was known and the purpose of the interceptions clear – to seize the initiative in a global war between countries who threatened each other's existence.
But in the years that followed, the enemies changed, intelligence capabilities developed, and the newly renamed GCHQ and newly formed NSA, created in October 1952, began an inexorable rise, intertwining interests and capabilities that insiders say would now be almost impossible to untangle. Professor Anthony Glees, who has written about this relationship, says it is one of Britain's last claims to global power status.
"In large part this position stems from three facts of British life, each directly connected in purpose to the other: our nuclear deterrent capability, our armed forces and our secret intelligence community," he said.
"Yet what gives us our critical mass as a power is one single, overarchingly important reality: our intense and intimate security relationship with the USA. Intelligence co-operation is its throbbing heart."
One senior member of Britain's intelligence community told the Guardian: "The relationship between the NSA and GCHQ is unique. Most intelligence agencies compete with each other. The CIA, for instance, sees MI6 as a competitor. They work with each other, but there is always some tension. The NSA and GCHQ are not like that.
"When you get a GCHQ pass it gives you access to the NSA too. You can walk into the NSA and find GCHQ staff holding senior management positions, and vice versa. When the NSA has a piece of intelligence, it will very often ask GCHQ for a second opinion. There have been ups and downs over the years, of course. But in general, the NSA and GCHQ are extremely close allies. They rely on each other."
This symbiosis developed during the attritional years of the cold war. It continued to evolve as signals intelligence – Sigint, as the agencies call it – became as important as human intelligence (Humint) – recruiting "moles" and informers.
And with the onset of the internet and cyberwarfare, GCHQ and the NSA, always in lockstep, achieved pre-eminence among their agency peers.
The Snowden files revealed intelligence-gathering is now being conducted on a grand scale, with the NSA and GCHQ exploiting advances in technology to tap into, store and analyse more and more information.
Without Snowden, we would not have known that the amount of personal data available to GCHQ from internet and mobile traffic increased by 7,000% between 2008 and 2012.
That is just what the UK collects; the files also revealed that 60% of all Britain's refined intelligence comes from the NSA.
From humble beginnings in rickety wooden huts, GCHQ has become the keystone of Britain's spy agencies, and its "doughnut" headquarters in Cheltenham is probably the most remarkable building ever constructed in the UK.
Covering more than 92,000 sq metres (1m sq ft), it is packed with supercomputers operated by codebreakers and data miners who work behind concrete and limestone walls that are up to 2.5 metres (8ft) thick.
With a staff of 6,400, it is the biggest of Britain's intelligence agencies. It is rather smaller, however, than the NSA.
At its headquarters in Fort Meade, Maryland, the NSA has more than 1,000 buildings on a 2,000-hectare (5,000 acre) site and employs an estimated 40,000 people. That figure does not include the analysts at different bases across the world, or the army of subcontractors needed to keep the agency running smoothly. Edward Joseph Snowden was one of them.
For a man who has been at the centre of worldwide attention for five months, surprisingly little is known about Snowden.
After "outing" himself in June, insisting he didn't want to hide behind a cloak of anonymity, he has spoken on just a few occasions, and only when circumstances seemed to demand it.
He avoided everyone he didn't want to see when he was in Hong Kong, the first place he escaped to, and for several weeks he remained beyond the reach of the world's media, and doubtless a small army of spies, while holed up in a hotel room in the transit area of Moscow's Sheremetyevo airport.
In the absence of regular pronouncements, there has been much animated speculation about why he did what he did, who has the material he took, and what kind of damage he has done.
And most of it remains just that – speculation, albeit of a kind that has fuelled character assassinations from those circling the wagons around the intelligence agencies.
Traitor was a barb he must have expected; he has also been branded a self-serving twerp (by the former head of MI5 Stella Rimington), a naive narcissist, and perhaps strangest of all, a cross-dressing Little Red Riding Hood (in the Washington Post).
What we do know about Snowden suggests he doesn't easily fit into any of those categories, or indeed, any stereotype. He does not look like a computer boffin, nor does he speak in the manner of a tortured ideologue.
There is no Julian Assange-like messiah complex for cod-psychologists to dissect, and money doesn't appear to matter much to him. He hasn't asked for, or received, any payment from the Guardian.
He remains something of an enigma, happy to stay out of the limelight.
Born on 21 June 1983, Snowden spent his early years in Elizabeth City, North Carolina. His parents, Lonnie and Elizabeth Snowden, were teenage sweethearts who met at Northeastern high school and married in a "back garden" wedding in 1979, when they were 18. They had a daughter, Jessica, and then Edward. They split up in 2001 and Lonnie, who retired from the US coastguard, has since remarried and moved to Pennsylvania.
In 1993, when Snowden was 10, the family moved to Crofton, Maryland, near the NSA's HQ. Neighbours who spoke to US newspapers said he was polite, quiet and seemed to spend too much time looking at computers. Dawn Whitmore, a former classmate, remembered him as a shy and serious boy, who sported a pudding-bowl haircut and thick glasses. "He was very well thought-out with what he was trying to say. He was always very, very nice with me, but I was also a very nerdy, shy girl."
Snowden went on to Anne Arundel community school, but dropped out in his second year. He dropped out again in 2004 but must have studied at home to earn a GED (General Educational Development) – the equivalent of a high-school certificate.
Snowden might have ended up in the army – he had four months in the reserves in mid-2004. "I enlisted shortly after the invasion of Iraq and I believed in the goodness of what we were doing," Snowden said. But he broke both legs in an accident and never completed his training.
Though he may not have had a string of formal qualifications, sitting in front of a screen honed something inside him, because in 2005, after a spell as a security guard, he was taken on by the CIA as an IT analyst. He was obviously good at it, because within two years he was one of the agency's operatives working in Geneva, Switzerland. It was not an experience he enjoyed.
"Much of what I saw in Geneva really disillusioned me about how my government functions and what its impact is in the world," he said. "I realised that I was part of something that was doing far more harm than good."
Snowden had a variety of jobs with confusing titles. Essentially the posts gave him privileged access to internal networks: ironing out problems, making systems work more efficiently and, ironically, making sure they were secure.
He was a troubleshooter, which is the principle reason why he managed to see so many documents – and spirit them away without leaving an electronic fingerprint.
"When you're in positions of privileged access, you're exposed to a lot more information on a broader scale than the average employee," he said. "And because of that you see things that may be disturbing. You recognise that some of these things are actually abuses. And when you talk to people about them in a place like this … people tend not to take them [the abuses] very seriously.
"Over time that awareness of wrongdoing builds up. And the more you talk about it the more you're ignored, the more you're told it's not a problem, until eventually you realise these things need to be determined by the public and not by somebody who was simply hired by the government."
In 2009 Snowden left the CIA to work in the private sector and four years later he got a job with Booz Allen Hamilton, a consulting company that supplies computer specialists to the NSA. Initially he was posted to Japan, then Hawaii. Snowden has admitted he only took the $122,000-a-year (£75,000) post to get access to certain material. By 20 May 2013, he had gathered what he wanted.
He boarded a flight to Hong Kong, leaving behind him a bewildered girlfriend, a boss who thought he needed time off to treat epilepsy (his cover story), and any chance he could ever again lead a normal life.
In interviews over the past four months, he has attempted to answer the questions that have been thrown at him, particularly by those who have made the most damaging claim – that he must have given his secrets to foreign governments.
He insists this is completely untrue. "This is a predictable smear that I anticipated before going public, as the US media has a kneejerk 'Red China!' reaction to anything involving Hong Kong or China, and is intended to distract from the issue of US government misconduct.
"Ask yourself: if I were a Chinese spy, why wouldn't I have flown directly into Beijing? I could be living in a palace petting a phoenix by now. No. I have had no contact with the Chinese government. I only work with journalists."
In his most recent interview in the New York Times, Snowden said he hadn't taken any secret files with him to Russia, where he has been given asylum for a year. "What would be the unique value of personally carrying another copy of materials onward? There's a 0% chance the Russians or Chinese have received any documents."
The four computers Snowden has been carrying with him since he left Hawaii were not jam-packed with secret files after all; they were a decoy. US officials who have been to Moscow have since said this is correct.
The other accusation made against Snowden is that he revealed secrets that would put people's lives in danger. He denies this emphatically. "I did not reveal any US operations against legitimate military targets. I pointed out where the NSA has hacked civilian infrastructure such as universities, hospitals and private businesses because it is dangerous. These nakedly, aggressively criminal acts are wrong no matter the target. The public needs to know the kinds of things a government does in its name, or the 'consent of the governed' is meaningless."
But the revelations must have affected national security? Think again, he asks.
"US officials say this every time there's a public discussion that could limit their authority. US officials also provide misleading or directly false assertions about the value of these programmes.
"Journalists should ask a specific question: since these programmes began operation shortly after September 11, how many terrorist attacks were prevented solely by information derived from this suspicion-less surveillance that could not be gained via any other source? Then ask how many individual communications were ingested to achieve that, and ask yourself if it was worth it.
Mastering the internet
Snowden was still a teenager when 2,996 people were killed on 9/11. The failure to detect the plot sent the intelligence services into panic; they recruited as many Islamic specialists and linguists as they could, redirected operations towards Osama bin Laden – and began the deliberate, elaborate process of building an intelligence machine that could feed information to sustain the wars in Iraq and Afghanistan.
It was the first glimpses of this evolving apparatus that so worried Snowden and led him – seven years later – to undertake the biggest ever theft of material stamped with the highest classification levels, called Strap 1 and Strap 2.
The wars against the terrorists coincided with a period of huge technological innovation, making it more difficult for the agencies to detect the important "noise" they were listening for amid all the chatter from the rest of us.
In Britain, the challenge thrown down by the explosion in use of the web, mobile phones and social media was met by a GCHQ programme that showed remarkable ambition. It was called Mastering the Internet (MTI).
In America, similar projects were just as audacious. The files released by Snowden show the agencies have been, and remain, determined to eavesdrop on every possible method of communication, regardless of how much extraneous material they gather in the process; they have put taps on the cables that carry raw internet traffic across the world; they have gone further "upstream" and devised ways of getting material from the computers which run the big internet service providers; they have refined their relationships with mobile phone companies so they can get details of every call made and received; and they found ways of defeating encryption software too, setting supercomputers to crack codes, or by inserting secret "back doors" into the software itself.
The scale of this technological achievement is admirable, the logic behind it clear; but all this impressive architecture has been built without any political discussion about whether this is the right thing to do, or any endorsement from millions of members of the public, whose personal lives are now being recycled through giant databases.
A sense of the anxiety that was driving GCHQ to do this was revealed in an internal memo, dated Tuesday 19 May 2009, which was written jointly by the director in charge of the MTI project and a member of the agency's cyber-defence team. The memo was a "prioritisation and tasking initiative" to another senior member of staff, who was being urged to come up with new ideas, fast.
"It is becoming increasingly difficult for GCHQ to acquire the rich sources of traffic needed to enable our support to partners within HMG [Her Majesty's government], the armed forces and overseas," they wrote.
"The rapid development of different technologies, types of traffic, service providers and networks, and the growth in sheer volumes that accompany particularly the expansion and use of the internet, present an unprecedented challenge to the success of GCHQ's mission." The memo continued: "We would like you to lead a small team to fully define this shortfall in tasking capability [and] identify all the necessary changes needed to rectify it."
The two chiefs said they wanted "potential quick-win solutions not currently within existing programme plans".
Those existing programmes might have been a reference to the NSA's Prism project, and Tempora, GCHQ's crown jewel, which was in development. The former was started in 2007 as a way for the NSA to get access to the computer systems which run Microsoft, Google, Facebook, Apple and other giants of the web world.
A 41-slide PowerPoint presentation dated April 2010 was among the Snowden documents, and it revealed the NSA was very pleased with the information it was receiving, which included search histories, the content of emails, videos, photos and live chats.
The NSA hailed Prism as "one of the most valuable, unique and productive accesses", and said it had generated 24,005 reports in 2012 (from a total of 77,000 over the previous five years).
Some of this information was being shared with GCHQ, though it is still unclear how much of this material was sought – requiring a warrant signed by a minister – rather than offered by the Americans, which would not.
Using secret court orders, the US was already bulk-collecting the telephone records of millions of US customers of Verizon, one of America's largest telecoms providers. With Prism, the NSA had another way of accessing vast amounts of information about people who were not under any suspicion at all.
But this was nothing compared with Tempora.
The British programme was far more ingenious, a real breakthrough in capability, albeit one which critics say meant an even bigger potential intrusion into the private lives of ordinary people.
GCHQ had been tapping the undersea cables that carry the internet in and out of the UK for years, but Tempora provided the ability to analyse the information in almost real time, rather than dumping the data into vast but simple electronic storage bins that take time to sift through.
The buffering that Tempora allows acts like Sky+ television, slowing down or briefly halting the stream of information to make it easier for other filters to search for keywords, names, or patterns of behaviour.
GCHQ keeps the content of messages for three working days, and the simple "metadata" – which includes details of who sent and received them – for up to 30 days. The programme was trialled at GCHQ's station in Bude, Cornwall (which is partly funded by the NSA) and was an instant success.
British delight at having scooped the Americans was matched by the NSA's desire to get its hands on the project; an internal US guide to using the system, which became fully functional in 2011, described it as "an exciting opportunity to get direct access to enormous amounts of GCHQ's special source data". When the NSA was offered access to Tempora for a trial period, the agency told its analysts to be on best behaviour.
"[We] need to be successful!," a memo urged. "We're depending on you to provide the business case required to justify expanded access."
"We need to prove that NSA's access is necessary to prosecute our mission and will greatly enhance the production of the intelligence. The success of this three-month trial will determine expanded NSA access to internet buffers in the future."
The strategy seems to have worked. By May last year, an internal GCHQ memo said it had 300 analysts working on intelligence from Tempora, and the NSA had 250.
The documents show Tempora gave the UK "the biggest internet access" of any member of the Five Eyes electronic eavesdropping alliance, comprising the US, UK, Canada, Australia and New Zealand.
They also show that by 2012 GCHQ was handling 600m "telephone events" each day. It had tapped more than 200 fibreoptic cables and was able to process data from at least 46 of them at a time.
With each cable carrying data at a rate of 10 gigabits a second, the tapped cables could, in theory, deliver more than 21 petabytes a day – equivalent to sending all the information in all the books in the British Library 192 times every 24 hours. "This is a massive amount of data!" one of the documents said. "You are in an enviable position – have fun and make the most of it."
All this activity was approved in the UK by a subsection of a law which was introduced in 2000 – the Regulation of Investigatory Powers Act (Ripa) – a time when the agencies could not have envisaged being able to conduct surveillance on such a massive scale, when buffering was not even possible. Ken Macdonald, the former director of public prosecutions, said these "blinding transformations" had rendered Ripa and other intelligence legislation "anti-modern".
"The spooks, who once sat in cubicles steaming open the glued-down flaps of a few dozen suspect envelopes, now have more fertile plains to furrow and the marvellous means to do it. Now they can steam open everything."
Mr Justice Michael Burton seems an amiable fellow: ruddy-faced and quick-witted, he is a specialist in commercial law and well known around the Inns of Court for his interest in amateur dramatics.
He is also the newly appointed president of the investigatory powers tribunal (IPT), which assesses complaints against Britain's intelligence agencies.
A speech he gave to lawyers over lunch on Monday 14 October was a first; no other head of the tribunal has ever spoken in public before, though Burton assured his audience in the City of London that he had no intention of marking this historic occasion by telling them anything of great interest. However, he did give a glimpse into the problems faced by the IPT, without offering any particular solutions.
"We do receive a large number of applications from individuals about their belief, or often their paranoia, that they are being targeted," the judge explained after finishing a roast beef sandwich.
"I am afraid we get quite a lot of complaints from members of the public who say, for example, 'When I was 14, I had my tonsils removed and I believe that MI5 implanted electronic equipment in me.' "Very often the sign is whether they are resident in a mental institution. It ranges from that to very serious complaints."
The tribunal is one of the cornerstones of the regime designed to scrutinise the agencies, a regime described by William Hague, the foreign secretary, as one of the best in the world. But Burton's court has taken a fair degree of flak over the years, for reasons he described himself.
"The media describes the IPT using terms such as 'a secret closed court', 'a little-known complaints body', 'one of the most secretive judicial bodies in the country', 'the UK's most secret court most of whose cases are held in closed session'," Burton said. "Almost all of that is untrue."
Really? The IPT is certainly unlike any other court; it does not publicise a list of when it is holding cases or where; almost all of its hearings are in private – there will be no public sessions for the rest of this year. And it almost always – in more than 99% of cases – fails to uphold complaints against the secret services or local authorities.
Snowden's documents show GCHQ declaring the tribunal has never ruled against a British agency since the court was established 13 years ago. The IPT will not say whether this is true or not. It is a secret.
The tribunal will also not even say where it is based. Its website refers to a PO box in central London. The Guardian tracked this down to a post office delivery office in the centre of an enormous building site near MI6 headquarters in Vauxhall, south London.
Apparently the IPT doesn't have a permanent home. In his speech, Burton acknowledged the court, which has eight part-time members, moves around various buildings in Whitehall. This is not done to bamboozle reporters or members of the public, he said, but because the tribunal is shunted around to where-ever there is free space.
The Snowden files suggest GCHQ is not unduly worried about the IPT; it is easy to see why. Over the past 13 years, it has heard 1,469 cases. It has upheld complaints against councils and police forces on 10 occasions – that's 0.68% of the total.
The figure is actually even lower because two of the successful complaints referred to the same matter in 2010.
The case in question illustrates another criticism often levelled at the IPT. It involved Poole council and its unlawful surveillance operation on Jenny Paton and her three children. It took the IPT more than two years to rule the council had no business using anti-terrorist laws to establish whether the Patons lived outside a school's catchment area.
But while there is some transparency in the way local authorities can use and abuse their power, there is nothing similar for the agencies.
This is the essential conundrum of the tribunal. How can people bring credible cases to the tribunal, if nobody knows what the agencies are doing – even in the broadest sense? With too little in the public domain, it is not surprising that many cases fall at the first hurdle for being "frivolous or vexatious".
When the IPT does decide to take on a complaint, it can ask for "all such documents and information as the tribunal may require". But this exchange relies on trust. The court has no way of checking whether it is getting what it needs. The panel will then make an assessment – in secret – before making a ruling, whose details will almost certainly never be published.
In eight years up until 2010, the IPT had only disclosed findings from five cases, a situation that even the police have been embarrassed by.
Giving evidence to parliament, the police lead on surveillance, Chief Constable Nick Gargan, admitted the IPT was largely anonymous and needed to be more transparent.
"[It] ought to be encouraged to be more publicly visible both in terms of encouraging people to use it and, where meaningful claims have been made, to actually publicise those findings," he said.
Burton said he was open to ideas about how the IPT could make itself more accessible, but only to a point. He said he wouldn't approve anything that meant "all-important secrecy was lost".
The parliamentary intelligence and security committee (ISC) is the other essential plank of oversight cited by the government, but it has also been bedevilled by criticism since it was established in 1994.
Too weak, too close to government, too reluctant to criticise the agencies are some of the recurring jibes. Earlier this year, it won new powers to force the agencies to hand over material, and a small increase in the staff to review it. But while doubling of its budget to £1.3m will give it more clout, it is starting from a low base.
With a team of part-time staff, the committee's nine MPs have a huge job to provide credible oversight of the three spy agencies, which have a combined staff of more than 10,000 and a combined annual budget of £2bn.
The ISC is now also responsible for reviewing the work of the military's "defence intelligence" unit, which will add to the mountain of documents available for review.
The committee's announcement on Thursday 17 October of a broad inquiry into surveillance, which will include some public hearings, is a watershed moment for the ISC, and its chair, Sir Malcolm Rifkind. He has insisted his team is capable of doing the job, but even former heads of the committee have acknowledged what a formidable task this is.
"The ISC is underfunded. It needs more people working for it," said Kim Howells, a former Foreign Office minister who chaired the committee for two years until 2010. "It needs more investigators and it needs to be able to call on expertise whenever it is required. It might be a no-brainer, but you try to persuade the Cabinet Office to part with the cash."
Howells also revealed how Gordon Brown tried to influence the ISC when he was prime minister, even though it is supposed to be entirely beyond this kind of executive interference.
"I had the most terrible battles with him and the head of the civil service. They didn't really understand the committee must be absolutely scrupulously independent of government and the agencies."
The former MP, who retired at the last election, called for parliament to determine whether the laws governing the agencies are "too swingeing and give too much leeway to executive action". "That is where the debate should be. And if parliament thinks there is too much power then parliament should change the law."
He conceded the job of the ISC becomes extremely difficult if it is "not trusted by either parliament or the media, and if people don't accept what it says. Then the suspicion of collusion starts to grow." But he defended the MPs who have served on the committee, saying they could be trusted to do a good job, if they had more support.
"Generally they are people who are either at the end of their political life or near the end, they have quit climbing the greasy pole, or been dragged off it."
Paul Murphy, an ISC chairman during Labour's years in power, remembered his committee had nobody to help them make sense of vast numbers of technical documents. He was in charge between 2005 and 2008 – when many of the current surveillance programmes were likely signed off by ministers. Yet Murphy and his team were often flying blind.
"We didn't have an investigator. You'd effectively have to rely on the word of the agencies and if there was any dispute you'd have to go through the documents yourself, which we did. Reams of it.
"The documentation is so vast now that the ISC may need more than investigator. It's not an easy position to fill and the person has to have carte blanche. Even then you have to be selective because it is such a huge task."
'The golden age'
Alan Turing remains the one true superstar of Britain's intelligence community. His feats during the second world war are now being celebrated in a way that was unimaginable when he died.
There are statues in honour of him in Guildford and Manchester, where he lived and worked, and he was the subject of a rare speech on 4 October last year by Sir Iain Lobban, the current director of GCHQ, to mark the 100th anniversary of the codebreaker's birth.
He rattled through the stories of Turing's peculiarities – burying his silver bullion and then forgetting where; chaining his mug to his radiator; cycling in his gas mask to ward off hay fever. Lobban credited Turing with starting the "irrevocable change" that led to the formation of GCHQ and its evolution into "the highly technological intelligence organisation that it is today".
He said that if Turing were alive he would be working on threats from cyberspace, a clever way of co-opting the codebreaker and his achievements into surveillance programmes that would have been inconceivable to him. "Our challenges come from the explosion in the volume of communications as well as the relentless increase in new ways of accessing and processing that volume," said Lobban. "Then, code related simply to the encryption of communications; today, code refers to the way in which we program IT systems. Then, the challenge was to identify German and Japanese communications; today, the challenge can simply be to cope with the number of different communications options. Today the internet provides the virtual global landscape for an analogous struggle."
But the struggle is different, and surveillance strategy has been turned on its head to deal with it.
Snowden's files revealed the mouth of the intelligence funnel has been stretched wide open over the last decade. Using technologies that are becoming more powerful and sophisticated, GCHQ and the NSA have been undertaking government-authorised data trawling; the secret services have quietly ushered in the age of the digital dragnet.
The agencies say they cannot do their work without these capabilities and they want to expand them; critics say they should not have been able to acquire them without a proper debate, and without more muscular accountability.
Perhaps it was this tension that led James Clapper, the director of US national intelligence, to concede that a fork in the road had been reached with the Guardian revelations, and not before time. "As loth as I am to give any credit to what's happened here, which is egregious, it's clear that some of the conversations this has generated, some of the debate, probably needed to happen," he said.
The arguments are delicately poised, the issues could not be more important: how to balance privacy and security in the 21st century. The Snowden files make it clear that GCHQ and the NSA have turned Turing's niche pursuit into intelligence-gathering on an industrial scale. An internal memo to analysts at GCHQ dated late 2010 summed up the mood about the powers now available to them: "We are in the golden age."
This week's podcast is devoted to an in-depth conversation between Alok Jha and the leading theoretical physicist Nima Arkani-Hamed about the future direction of physics and some of the baffling questions it has yet to answer. Alok and Nima are joined by the science writer Graham Farmelo, author of The Strangest Man, about theoretical physicist Paul Dirac.
Graham and Nima recently took part in a day of discussions to mark the opening of the Collider exhibition at the Science Museum in London.
Going beyond the discovery of a Higgs boson at the LHC earlier this year, Nima introduces the next phase of Cern's work and explains why his certainty about the existence of the Higgs is not matched when it comes to supersymmetry.
We're always here when you need us. Listen back through our archive.Alok JhaJason Phipps
I'm in Houston at the moment, training with Nasa for a mission to the International Space Station. I'll be blasting off in 2015! Before I saw this film, about two astronauts on a not entirely dissimilar mission, my friends warned me not to take it too seriously – since the astronauts end up floating in space. So I took their advice and loved it.
I found the scene where Sandra Bullock flips out of her spacesuit in a couple of seconds very amusing. Anyone who's ever worn one knows that's impossible: it takes about three minutes just to wriggle into the upper torso section. And as for what she's wearing underneath: well, we certainly don't wear anything that sexy! We're in long-johns, a cooling suit covered in narrow tubes filled with water, and a nappy – since a spacewalk usually lasts around eight hours and you have to stay well-hydrated. But showing Bullock in a nappy probably wouldn't sell so many tickets.
It's unlikely, too, that her character, an engineer sent to attach some device to the Hubble telescope, would be on a spacewalk after just six months of training. You can certainly go into space then: Sarah Brightman, for instance, is coming with us in 2015, and will have notched up that amount of time. But at that level, you're very much a space tourist rather than a fully fledged astronaut.
The shuttle that Bullock and veteran spaceman George Clooney use to get to the ISS has actually been retired: the only vehicle we have to get there and back today is the Russian Soyuz rocket. The film does capture the essence of what it's like to be in zero gravity, though: how difficult it is to move; how you have to struggle to stop yourself from just rotating endlessly. And the vistas of Earth and space are stunning, very much like the movies you can see on YouTube made of time-lapse photographs taken from the ISS.
None of the astronauts I know have been put off going into space by seeing Gravity. We all know the risks. And the real dangers can be much worse: my spacemate Luca Parmitano had his helmet fill up with water while on a spacewalk. It was just about the worst disaster Nasa had ever had at the ISS. Like Clooney, you do have to keep a cool head when you go out into space. You never know what might happen.More on GravityLaura Barnett
Each day in the run-up to Christmas, we present an extract from Barnaby Rogerson's fascinating Book of Numbers. Today, our festive countdown reveals that any discipline can be boiled down to two basic principles
 People love to play the Two Things game, but rarely agree about what the two things are.  That goes double for anyone who works with computers.
A few years ago, Glen Whitman was chatting with a stranger in a Californian bar. When he confessed to this stranger that he taught economics, the drinker replied without so much as a pause for breath, "So what are the two things about economics? You know, for every subject there are really only two things you really need to know. Everything else is the application of those two things, or just not important."
"Okay," said the professor. "One: Incentives matter. Two: There's no such thing as a free lunch."
Inspired, Glen started playing the Two Things game and recording some of the results on a web page (Google "Whitman" and "Two Things" and you'll get there). But it's more fun to try it for yourself – and especially good if you find yourself at a dinner next to a self-important professional. Here are some of the best of Whitman's:
Finance:  Buy low.  Sell high.
Medicine:  Do no harm.  To do any good, you must risk doing harm.
Journalism:  There is no such thing as objectivity.  The end of the story is created by your deadline.
Theatre:  Remember your lines.  Don't run into the furniture or fall off the stage.
Physics:  Energy is conserved.  Photons (and everything else) behave like both waves and particles.
Religion:  Aspire to love an unknowable god.  Do this by trying to love your neighbour as much as yourself.
• Taken from Rogerson's Book of Numbers by Barnaby Rogerson (Profile).Barnaby Rogerson
Reports of the death of Comet ISON as it brushed within 1,165,000 km of the Sun last Thursday were premature. Something survived, but as I write this we don't know whether ISON still has an intact and productive icy nucleus or whether we are left with only a flying bank of rubble that may dim and disperse within days.
I am hopeful that the comet's tail may be seen stretching vertically upwards from our ESE horizon before dawn over the next few days, and also slanting up and to the right in the W after dusk. Whatever remains of ISON is now climbing northwards against the stars to pass midway between Vega and Arcturus on the 21st and stand at its closest to the Earth, 64m km, on the 26th. Whether it will be bright or quite invisible is anyone's guess at present. Anticipate an update (or inquest?) here on the 16th.
Venus is at its brilliant best (mag -4.7) as it stands near the young Moon on the 5th and 6th. Find it low down between the S and SW as the evening twilight fades. It swells to 59 arcsec, so its slim crescent is obvious through binoculars.
Jupiter rises in the NE at about 18:30 at present and soon after sunset by the year's end, climbing to cross the high meridian after midnight. Conspicuous at mag -2.6 to -2.7, it is retrograding (tracking westwards) to the S and W of Castor and Pollux in Gemini.
Mars improves from mag 1.2 to 0.8 as it moves E in Virgo. After rising in the E at about 01:00, it climbs into the S before dawn. Saturn, brighter at mag 0.6 in Libra, stands low in the SE before dawn at present, but rises more than 4 hours before the Sun by the month's end. Mercury is mag -0.7 and lies just above the SE horizon as the morning twilight brightens over the next few days.
Moonlight swamps all but the brighter meteors of the Geminids shower which lasts from 8th to the 17th and peaks before dawn on the 14th. Its slow meteors diverge from a point some 10° to the N of Jupiter.
3rd 00h New moon
6th 00h Moon 8° N of Venus
9th 15h First quarter
14th 01h Peak of Geminids meteor shower
17th 09h Full moon
19th 07h Moon 5° S of Jupiter
21st 17:11 Winter solstice
25th 14h Last quarter
26th 03h Moon 5° S of Mars
29th 01h Moon 0.9° S of Saturn; 06h Mercury in superior conjunctionAlan Pickup
We are used to thinking of TB as a disease of the past, but this killer is back and western leaders need to act now
It has killed kings and poets. Henry VII fell not in battle but of a feared disease, consumption. Keats, Kafka and Chekhov were victims. When La Traviata and La Boheme first played in the opera houses of Europe, the portrayal of young women coughing their last breath was unremarkable. The disease accounted for a quarter of all deaths.
By the end of the 19th century, the bacteria causing what we now call tuberculosis (TB) had been identified and patients were consigned to sanatoriums. Later the BCG vaccine was developed and the discovery of antibiotics allowed successful treatment. Archbishop Desmond Tutu and Pope Francis are survivors.
So that's the end of the story, right? TB is beaten, a disease of the past. Wrong. TB has made a terrible comeback. Always a disease of poverty, now linked to HIV-Aids, it is killing 1.3 million people a year worldwide, not just in sub-Saharan Africa and India, but in eastern Europe, too. Two decades ago, the World Health Organisation declared it a global emergency. And, actually, there never has been an effective vaccine. BCG only works for children, for a limited time, and offers negligible protection against the most common forms of TB.
Now this oldest of diseases is producing the newest of threats. In 2006 53 patients in a rural hospital in South Africa were found to have contracted a highly drug-resistant form of TB: 52 of them died. Drug-resistant strains now account for a third of all deaths from the disease.
Earlier this year, a Time magazine cover sensationally declared "Contagion: why drug resistant TB threatens us all". Migration has translated TB from an international development cause to a domestic public health challenge. London has the highest rates of TB of any city in western Europe. The borough of Newham has rates equivalent to Nigeria.
Those with access to advanced healthcare who contract drug-resistant TB face a long and extremely unpleasant course of treatment, but stand a chance of living. For those in less developed countries, it is usually a death sentence.
It's unsurprising that drug resistance should have occurred. TB is treated with drugs developed over 60 years ago. Long courses of antibiotics, administered in patchy or non-existent healthcare systems, where counterfeit pills are rife, make non-completion of drug regimes a constant risk.
Indeed, the entire apparatus to control TB in high burden countries is pitifully antiquated. Diagnosis can take weeks as samples are posted off to laboratories. If this had been a disease that had resurged in the west, we would by now have a new vaccine, rapid testing and better drugs.
But there was no commercial market for these life savers, and so the pharmaceutical companies had no interest in developing them. Only the intervention of the west through massive aid programmes and partnership funding for research can change the story.
It is beginning to happen. The rate of new cases of TB has been falling worldwide for about a decade, enough to hit a UN millennium development goal target, and deaths will have nearly halved since 1990. But a decline of 2% a year in the estimated incidence rate suggests that the disease is being beaten at a shamefully slower rate than when the west tackled it a century ago. On current progress it will take at least another 100 years. The latest World Health Organisation report, published last month, warned that 3 million people a year who develop TB are being missed by health programmes. Most worryingly, less than a quarter of drug-resistant cases are being detected and less than half of those that are detected are successfully treated.
Political commitment and new resources are needed. The Global Fund provides over 80% of all international financing to fight TB. So its replenishment, to be agreed in Washington this week, is crucial. To its credit, the UK has announced that it will step up support. Other wealthy countries must follow.
It's hard for western leaders to commit money at a time of austerity. But quite apart from the moral obligation, TB is a disease that does not recognise national borders. We have a common interest in fighting it. The rising treatment costs of inaction, particularly in respect of drug-resistant TB, argue for intervention now. And with a new vaccine, modern diagnostics and advanced drugs all in sight, we could change the trajectory of the disease for good.
Environmental tragedy in the Philippines has killed thousands of people and rightly captured attention. The tragedy of TB is that it is claiming millions of lives, yet no one is talking about it. It's time to tear consumption away from the pages of 19th-century novels and shout to the world that this killer is back.Nick Herbert