By Will Douglas Heaven
Far from breaking it, the surge in usage the internet is seeing right now is driving a major upgrade.
Just like that, our internet connection has become an umbilical to the outside world. We now depend on it to do our jobs, to go to school, and to see other people. It is our primary source of entertainment. And we’re using it a lot.
Between January and late March, internet traffic increased by around a quarter in many major cities, according to Cloudflare, a US company that provides network infrastructure to businesses around the world. Demand has skyrocketed for certain online services in particular. Video calls have replaced face-to-face interaction with colleagues, family, and friends alike. More people started using the video-conferencing software Zoom in the first two months of 2020 than in all of 2019. Stay-at-home entertainment is also booming. Record numbers of people are using Steam, a popular online PC game store. At one point this weekend more than 24 million players were logged on at the same time, a 25% jump since February. And online grocery stores are unable to handle the surge in business, with customers waiting for hours in virtual lines tens of thousands of people long.
So how is the internet coping with the most sudden burst of usage in its history? There are understandable signs of strain: Wi-Fi that slows to a crawl, websites that won’t load, video calls that cut out. But despite the odd hiccup, the internet is doing just fine. In fact, the covid-19 crisis is driving the biggest expansion in years.
“Anecdotally, the internet is struggling to keep up with the shift,” says Matthew Roughan at the University of Adelaide in Australia, who leads a mapping project called the Internet Topology Zoo. “You tend to hear the bad-news stories at the moment.” Paul Barford at the University of Wisconsin, Madison, who runs another internet-mapping project, agrees. The more we use the internet, the more we notice its glitches. Still, you might see brief, local disruption but not broader effects, he says: “That’s the whole point of a distributed network.”
As well as using the internet more overall, we are also using it at different times and in different places. Instead of an after-work peak at 7.30 p.m., there is now a peak just before lunch, says Cloudflare CEO Matthew Prince. It is not clear why this is, but late morning may be when a lot of people are in virtual meetings or classrooms. These shifts are not universal. According to Cloudflare, internet use has increased by around 40% in Italy. But in South Korea, where people were already heavy internet users throughout the day, the change is less pronounced.
Cloudflare also has data on where we’re connecting from. It has produced maps that reveal how human activity has left city centers behind and decamped to the suburbs. The maps show the change in daytime internet usage between Wednesday, February 19, and Wednesday, March 18, before and after large numbers of people started working from home. Urban hubs are red, indicating a decrease in internet use, surrounded by rings of green, indicating an increase.
Change in internet use in London (left) and New York (right) between Wednesday 19 February and Wednesday 18 March. Red shows a decrease in traffic, green shows an increase. (Cloudfare)
The sudden spike in demand is being taken seriously. After being lobbied by Thierry Breton, the European Union’s commissioner for internet markets, a raft of video-streaming companies, including Netflix, YouTube, Facebook, and the newly launched Disney+, all agreed to cut the picture quality of streamed video in Europe to avoid adding to the strain. Video accounts for more than half the internet’s traffic, according to analysis by internet hardware firm Sandvine. Similarly, Sony, Microsoft, and Valve, which runs Steam, have cut back on sending out updates to video games or restricted them to off-peak hours.
In addition to the increase in traffic, sheltering in place strains the internet in two more ways. First, last-mile connections—the ones that run from local exchanges or data centers to your home—are typically the weakest links in a network. Many run over outdated cables. When broadband was rolled out in the US, for example, it often piggybacked on cables originally installed for TV. These cables were designed to pipe data into a home and not out of it, which is why uploading video from a home internet connection can be flaky. Home connections also tend to have lower bandwidth than those in an office or school, making familiar activities feel slower. And the narrower pipes at the internet’s fringes get congested when everybody in a neighborhood wants to use the internet at the same time.
A second issue is that internet companies are now having to handle traffic from multiple locations instead of a handful of hubs. For example, it is easier for Dropbox to manage a thousand users when they are all connecting from a single university campus or office building. All of that activity can be piped over a single high-speed connection. But now those thousand users are spread across the country and connecting from hundreds of different networks.
Still, despite these niggles, the internet seems to be doing just fine. Health checks from RIPE and Ookla, two organizations that monitor connection speeds around the world, show minor slowdowns but little change overall. When Netflix and YouTube throttled back, it wasn’t because the internet wasn’t coping, says Prince: “Volunteering to do this in advance of any problem shows they’re good internet citizens.” It’s a just-in-case measure.
We may have an industrial revolution to thank. Two decades ago there was little commercial interest in the internet, which meant its infrastructure was managed in a relatively ad hoc way. A one-off event, such as a major news announcement, could make everything crash, says Tesh Durvasula, CEO of CyrusOne, one of several international companies that help keep the internet running by installing and managing the vast clusters of computers that make up the cloud. Now, a whole industry has sprung up and remade the internet for its own ends. Telecom companies like Comcast, content makers like Netflix, retail giants like Amazon, virtual storage providers like Dropbox—plus a host of other data centers and cloud services—have piped in new connections, shored up old ones, and wired up millions of superfast servers. The investment has massively increased capacity, speed, and performance across the board. “The industrialization of the internet has created a powerful mesh of networks that is, for the most part, working beautifully,” says Durvasula.
Built With Purpose
Cloud services let companies like Netflix and Dropbox scale up their operations quickly. When demand spikes, extra servers from the pool are corralled into use. Many internet companies use dedicated cloud platforms like Amazon Web Services (AWS), but the biggest are starting to run their own data centers, giving them even more flexibility to meet users’ needs. What’s more, streaming and gaming services have started to create purpose-built delivery networks that bypass large sections of the public internet. Traffic flows from their own data centers through their own cables to last-mile broadband providers like Comcast or AT&T.
Just as massive investment in a city center leads to a landscape transformed by construction projects, investment in the internet has led to similar expansion. It is harder to see, but it has completely changed our online experience. And it is why the internet is holding up so well right now.
In fact, far from bringing networks to their knees, covid-19 is driving the most rapid expansion in years. To make sure they meet demand, internet giants like Netflix and Equinix, which operates 200 data centers around the world, are rushing out upgrades as quickly as possible. Equinix is in the middle of upgrading its traffic capacity from 10 to 100 gigabytes. The work was going to have been carried out over a year or two—but it is now being done in a few weeks.
Netflix is expanding its infrastructure too. It typically partners with the largest data centers in a region, which stream local copies of Netflix’s catalogue into people’s homes. Being as close as possible to viewers makes delivery far faster. The company is now looking to install hundreds of extra servers in the second and third biggest hubs in each region as well, says Dave Temkin, vice president of network and systems infrastructure. Zoom is also trying to get closer to its users. It is monitoring where most of its traffic comes from and partnering with broadband providers in those locations to set up dedicated connections.
Cable companies are adapting their services as well. In the US, Comcast has waived data caps, for example. If the experiment is successful, the waiver could become permanent. And in South Africa, where millions of people are able to access the internet only via mobile phones, the government is relaxing regulations to allow network operators to roll out cheap or even free data plans. It is also opening up sections of the radio spectrum previously reserved for TV to internet traffic. Some of these measures may be undone when the crisis is over, but others will outlive it. Once cut, red tape is hard to stick back together.
There are still a handful of stumbling blocks that could slow the upgrade. For example, crucial parts of the internet still require a human touch. There are always cables that need replacing or servers that need fixing, jobs that only human engineers can do. For Barford, this could be where we start to see problems. “If those people are no longer able to get out into the field, then there may be local outages that persist,” he says.
If supply chains break down, internet companies may also find it hard to get hold of equipment and hardware. China is the largest producer of optical fiber and other essential hardware, such as semiconductors, for example. An interruption in supply could mean that plans to install new broadband connections to rural parts of the world are put on hold.
That could be bad for people in poorer regions who now find themselves cut off more than most. But overall, the internet is emerging stronger than ever. No other utility—not electricity, not water, not transportation—could handle such an increase in usage, says Prince: “The internet was built for this.”
Originally published by MIT Technology Review