RidE A black swan

If you thought the coronavirus was causing havoc, just wait until we have a digital network infection.

From virus-ravaged central London, the rebel Tory MP Sir Oliver Letwin, one-time UK Cabinet member and Minister for Brexit, is breathily talking into his smart-phone about a bigger threat to humanity than Covid-19.

What if the virus wasn’t targeting humans but instead infecting the digital networks we rely on for our essential services – electricity, communication, water, hospitals and transport.

The saving grace of our current Government-mandated Level-4 lockdown is that essential services continue to function. We work from home hunched
over our laptops making glitchy internet video calls to our colleagues.

Hospitals remain open, albeit clogged with patients reporting flu-like symptoms and the sharemarket continues to trade, even if KiwiSaver
investors would probably favour a lengthy trading halt.

The coronavirus disease will continue to spread and may kill thousands who otherwise would have lived. But in Apocalypse How? Technology and the Threat of Disaster, a book that’s both parable and pointed critique of society’s inability to adequately deal with risk, Letwin envisages a much
shorter, sharper shock that could ultimately prove far more catastrophic.

Network of networks

The growing problem, he says, is our reliance on converged networks that are increasingly complex in nature and more interconnected than ever. The fibre optic network and 5G mobile base stations that deliver increasingly fast internet access are also the basis of the next generation of smart electricity grids, remotely controlled power stations and intelligent
transport networks.

We got a flavour of that vulnerability this week when lockdown measures led to a 350% spike in call traffic on Spark’s network, overloading it and leading to patchy phone service. The telco, which recently focused on boosting capacity in its data network, recommended we use internet calling services such as Whats-App and FaceTime, instead. An outage of the data network, which increasingly carries voice traffic, would cause

Software applications that run complex systems are increasingly stored in the cloud computing systems that could be physically hosted in San Francisco, Sydney or Shanghai.

“We’ve collectively become dazed by the extraordinary brilliance and effectiveness of these technologies, and we’ve come to rely on them, all of us, increasingly, because they do so much for us,” says Letwin, a 63-year-old
Etonian who worked in Margaret Thatcher’s policy unit during the painful years of the mid-1980s economic reforms, before winning election himself as the Conservative MP for West Dorset in 1997.

By 2037, which is when the fictional scenario he paints in Apocalypse How?is set, a network of networks, privately run though regulated by governments, is also likely to control autonomous vehicles and extend to communication provided by constellations of satellites orbiting the Earth, controlled by the likes of Elon Musk, whose StarLink satellites are already occasionally visible as a pinprick of lights across the night sky.

“Over the next 10 to 30 years, we are going to see the cars we are used to driving go electric and we are going to see them driven much of the time for us by this network of networks,” Letwin predicts.

“The world is not as well-prepared for biological risks as it should have been, but it’s actually more prepared, alarmingly, for biological risks in many respects than it is for this technological risk.”

Crash override

It may not be some sort of malicious infection that takes the network down, though there is a precedent for that. In 2016, the week before Christmas, a transmission station in Kiev was shut down, cutting
electricity supply to about a fifth of the Ukrainian capital.

Researchers found that the power outage, which lasted for around an hour, was the result of sophisticated malware being uploaded into the software system that controlled the electricity substation.

They dubbed the malware Crash Override, and it’s one of the few examples so far of malicious software being designed to automatically infiltrate physical infrastructure with the purpose of disabling it. The security community believes Russian hackers are behind it. Similar attacks on Ukrainian power stations the previous year were more rudimentary in nature and required hackers to manually switch off circuit breakers using remote software tools.

It was almost as though someone was doing a dry run for a more extensive operation. The Ukrainian blackouts were considered a wake-up call, and governments around the world have beefed up cybersecurity measures in recent years to protect critical infrastructure. Our government’s Cortex system is a collection of cyber-threat detection and disruption tools run by the National Cyber Security Centre (NCSC).

They are applied under various acts of Parliament to government departments and operators of critical national infrastructure, such as the power grid and broadband networks, as well as key exporters and research institutions. In the year to June 30, 2019, the NCSC recorded 339 cybersecurity incidents and estimated it reduced harm to networks to
the tune of nearly $28 million.

But it isn’t just malicious attacks from cybercriminals or nation states that Letwin fears. Natural disasters could equally trigger the network collapse he envisages. In Apocalypse How? disaster strikes in the form of space weather.


The book opens on New Year’s Eve with Aameen Patel starting the late shift behind his traffic control computer terminal at Highways England’s Swindon HQ, where he will see in 2038.

Near midnight, his screen goes blank and the lights in the control room blink off before coming back on as the emergency generator kicks in. But glancing out of the control centre window, which overlooks the
M25 motorway, Patel sees to his horror a “growing pile-up of cars and lorries, partially illuminated by headlights jutting upwards and outwards as the concertina accumulated on the icy road”.

Something has gone terribly wrong with the country’s autonomous vehicle control system, which runs on the same communications system that underpins the internet as well as the country’s electricity, gas and water-control systems. The fictional sections of Apocalypse How? read like an Ian McEwan thriller, albeit without the literary polish. They follow Bill Donoghue, an official at the Bank of England, as he tries to juggle care for his elderly mother-in-law with his responsibilities to the country. He joins crisis meetings in the Cabinet Office, where an incredulous prime minister tries to keep control of a country plunged into darkness in the depths of winter.

It’s familiar territory for Letwin. After the 2015 general election returned David Cameron and the Conservatives to power, the MP for West Dorset was given responsibility for the Cabinet Office after years of serving as the Minister for Resilience. “During that time I conducted a series of reviews of
the resilience of the UK to different types of risk,” says Letwin.

He dealt with the usual problems such as floods, expected to increase in frequency due to climate change, petrol strikes and the effects of severe winter storms. “But it became clear to me that the two things we were most exposed to, ironically, were biological risk of the kind we are currently
in and technological risk arising from convergence.”

No backup

By 2015, Letwin was handling, on behalf of Downing Street, the UK’s response to the Ebola crisis in West Africa, including helping to smooth the way for the development of an experimental vaccine.

“The coronavirus doesn’t have anything like the mortality rates that Ebola had,” he says. “And as we have discovered, it has a much greater propensity to spread.” All the time, he was mulling over what he saw as the bigger threat to humanity, the convergence of digital networks.

“It’s not just a network of networks increasingly converging in one country. It’s a network of networks increasingly converging globally.”

Undersea cables connect countries with high-speed links. Data centres clustered around the world host data with little regard for international boundaries. Already, some New Zealand government services are hosted on Microsoft’s Azure servers based in Canberra. In Europe, natural gas is piped across the continent from Russia, supplying nearly 40% of the European Union’s needs.

Over coming decades, the push for efficiency will compel infrastructure operators to run the data layer and control systems for all of the above over the same network or a small subset of networks. Letwin sees this as inevitable and not something to fight against. Doing so would cripple countries’ competitiveness. “There’s no point in trying to resist it. There’s no other way to proceed than by accepting that a single network of networks is more efficient and produces more benefits to mankind than a disaggregated set of systems will.”

But he sees the escalation in cybersecurity measures as ultimately ineffective against the rare type of digital disaster he fears the most – the highly unpredictable and potentially disastrous events that the Lebanese scholar Nassim Taleb dubbed black swans. There will probably always be a chink in defences that allows a hacker in or an over-looked vulnerability to allow a cascading failure to tear through a highly converged network hit by a natural disaster.

Letwin’s space-weather scenario, in which solar flares and radiation storms
knock out energy and communications systems, may sound far-fetched. But it is included in the most recent update of the UK Cabinet Office’s National Risk Register of Civil Emergencies as a serious threat that the Government is trying to plan for, alongside the more familiar sources of
trouble, such as terrorist attacks, volcanic eruptions and floods.

Nor is Letwin the first to sound the dangers of convergence. Experts have warned for years of the dangers of highly complex and tightly coupled technological systems going awry.

Why systems fail

In their 2018 book Meltdown: Why Our Systems Fail and What We Can Do About It, Chris Clearfield and András Tilcsik explore the growing complexity of the systems we rely on in all aspects of life.

“From jet engines to home thermostats, billions of new devices are now part of a network called the Internet of Things, a huge, complex system vulnerable to both accidents and attacks. Complex computer programs are more likely to have security flaws,” they write.

“Modern networks are rife with interconnections and unexpected interactions that attackers can exploit. And tight coupling means that once a hacker has a foot-hold, things progress swiftly and can’t be undone.”

One example is Kiwi hacker Barnaby Jack, who gained notoriety for demonstrating how to hack ATMs, pacemakers and even insulin pumps – before his death from a drug overdose in 2013. With opaque algorithms increasingly built into networks to shape data traffic or intelligently monitor usage, the potential points of failure have increasingly more to do with computer code than water pipes or electricity cables.

Buy high, sell low

Clearfield is a former derivatives trader and Tilcsik a business professor at the University of Toronto. They pluck from the financial world an example that perfectly illustrates their thesis. In 2012, New Jersey-based equities trader Knight Capital Group suffered a bizarre trading glitch that saw it lose US$460 million in one hour.

A problem with the company’s high-velocity trading software caused its computers to put in a flurry of orders for shares on the New York Stock Exchange as the market opened on the morning of August 1. The automated system that allowed Knight to make thousands of
simultaneous orders and trades, giving it a competitive edge, now saw millions snapped up in unwanted trades in the space of 45 minutes.

Unwinding the unwanted positions nearly destroyed the company, with a group of investors having to inject US$400 million to save it. Knight was bought soon after by rival trading firm Getco. The problem was down to human error. One of Knight’s software engineers had upgraded the trading software but neglected to copy it to one of the company’s computer
servers, which ended up running old test code that simulated buying stocks at high prices and selling them cheap – the opposite of rational trading behaviour.

A lack of red flags and safety alerts built into the system allowed the software to run in real-life conditions. It was never intended as anything other than a dummy experiment to test Knight’s algorithms.

Letwin, who worked in investment banking in the 1990s before entering Parliament, sees the same issue of high complexity and tight coupling endemic in the banking sector that sparked the financial crisis of 2008.

“It just didn’t occur to them that this might happen or that it might be triggered by relatively minor problems in one part of the global system gradually affecting the rest of the system,” he says of the derivatives
trading that caused the banking collapse and which wasn’t identified adequately as a major risk in banking regulators’ models. The authors of Meltdown urge us to look to the commercial aviation sector for inspi-
ration on how to design systems to avoid disaster. It doesn’t get more high-stakes than flying thousands of people in expensive aircraft every single day.

Fallback provisions

Even when a plane’s automatic navigation and steering systems are
disabled, pilots can still safely control it, because of the numerous fall-
back provisions built into the plane’s design and refined by the likes
of Boeing and Airbus. It is these provisions, the equivalent of the torch
in the bedside drawer for when the power goes out, that Letwin
says we need to focus on to help us get through the short but critical periods when our high-tech networks fail.

What he’s actually proposing would probably have network engineers in
stitches of laughter. “We need to spend some money, not colossal amounts but some money, providing ourselves with low-tech analogue fallbacks that will see us through.”

He’s talking about stores of paper maps in case the GPS (global position system) satellites are taken out, filing cabinets full of patient records so hospitals can still function in a digital blackout, and diesel generators to power the growing network of electric-car charging stations if the power grid fails.

“Having an insurance policy is not wildly expensive, because you don’t
have to create new things,” says Letwin. We need to “maximise divergence
rather than convergence” when it comes to these networks of last resort,
he argues. That’s about as sophisticated as his solution gets and it’s that way by design.

He’s the first to admit that bureaucrats are terrible at planning for disaster. The typical cost-benefit analysis exercise used to justify public spending doesn’t cope well with extremely rare black-swan events such as catastrophic network failures with complex impacts that are hard to put a cost on.

“It is no surprise that in many countries, the diffusion of responsibility has been associated with what has every appearance of inaction,” says Letwin.
“There are quite enough people for each to be able to blame the others if and when disaster strikes.”

But unlike with our ineffective efforts to deal with climate change, there’s a
competitive aspect to network resilience. No country will want to go dark in a natural disaster or be blackmailed by a cyber attack. National self-interest will push us towards developing analogue fallbacks.

Letwin just hopes we don’t face catastrophe first. Apocalypse How? ends with the fried networks being repaired and revived. But after five days of starving in the cold, nearly 100,000 mainly elderly Britons are dead. Letwin,
thankfully, is more optimistic about our prospects of dealing with Covid-19.
As he and I spoke, British Prime Minister Boris Johnson, whom Letwin successfully defeated in an effort to prevent a no-deal Brexit in the faction-riven days before the general election in December, was announc-
ing stricter social distancing provisions to try to slow Covid-19’s spread.

“If the governments of the world are sufficiently radical in their approach to injecting very large monetary and fiscal stimulus into economies that would otherwise suffer massive declines, we can probably get through this,” says Letwin, who after 23 years in Parliament ended his political career as an independent at odds with many of his colleagues over Brexit. These days, he enjoys having time to work on book projects as a senior fellow at right-leaning think tank the Legatum Institute.

If Brexit division cast a pall over the final years of Letwin’s parliamentary career, he doesn’t see the UK’s split with the European Union as increasing the country’s vulnerability to the disaster he imagines in Apocalypse How?

“These issues are rising because of global technological trends. We’ll all be exposed to them long after we finish talking about Brexit.”

APOCALYPSE HOW? Technology and the Threat of
Disaster, by Oliver Letwin (Allen & Unwin, $39.99)