91av

THE modern world is drowning in data. In 1984, the global traffic of the fledgling internet amounted to 15 gigabytes per month. By 2014, that had become the average traffic per user. In 2019, each of us burned through that much data in just over a week. The floodgates show no signs of closing, either. As billions of new users come online, and ever more devices become web-connected, the amount of data in the world is forecast to rise to 175 zettabytes (10²¹ bytes) by 2025 – more than three times humanity’s output to date.

Processing these oceans of data requires enormous infrastructure, extending beyond smartphones and personal computers to millions of energy-hungry data centres around the globe. That combined hum already uses 6 per cent of the world’s electricity, an energy bill predicted to double by 2030, raising concerns about the sustainability of our digital habits.

For decades, technological improvements kept the rising waters at bay, allowing hardware to get smaller, faster and more energy efficient. But the silicon chips we rely on are starting to hit physical limits, threatening to leave us with an energy bill we can ill afford to pay.

“175 zettabytes
the total amount of data forecast to exist by 2025”

A plethora of alternative technologies are vying to continue the upward march in processing power, but most are still languishing on the lab bench. That is why a growing number of researchers are calling for something more transformative: a complete rethink of the thermodynamics underpinning computing. If the idea gains traction, it could revolutionise how computers are designed, allow processors to grow more powerful without huge extra energy demands, and sate our growing appetite for data.

Anyone who has ever sat with a laptop on their knees knows computers give off heat. A lot of heat. That is an unavoidable consequence of how they work, and to understand why, we have to think about what computers do. Broadly speaking, they are machines capable of storing and manipulating information. Computers do that in the form of bits: fundamental units of digital information that can adopt one of two states, either a 0 or a 1. These are represented in computers by tiny electronic switches called transistors that flick on and off when a voltage is applied. This process generates electrical resistance in computer chips, which manifests as heat. Given that modern computer chips feature billions of transistors working together, this can raise the temperature considerably.

In 1961, IBM physicist Rolf Landauer set out to calculate the theoretical efficiency of a perfect computer – one that wasted none of its energy in combating resistance. He knew that such a device would still consume some energy. That is because, like all machines, computers are constantly fighting one of the most powerful forces in the universe: the second law of thermodynamics. This states that the disorder of any closed system – a measure known as entropy – will always increase. It is why eggs don’t unscramble and marbles are easier to spill than clean up. As anyone who has tried to tidy up after their children knows, restoring order costs energy.

“6 per cent
the share of the world’s electricity currently devoted to computing”

Landauer reasoned that corralling information is a rebellion against disorder and so represents a decrease in entropy that can only be bought with energy. He worked out that even the simplest computation possible – erasing a single bit – must incur a tiny thermodynamic debt, no smaller than 2.8 × 10^-21 joules. Operating at this efficiency, Summit, the world’s most mighty supercomputer, could be powered by a few milliwatts. In reality, it uses 13 megawatts, the approximate peak output of two offshore wind turbines.

Even this is astonishingly efficient compared with early computers. The Cray 1 supercomputer, unveiled in 1975, used roughly 1 per cent of that power, but had less than a billionth of the computational muscle. The shift to where we are today was enabled by two remarkable trends working in tandem. The first was industry’s ability to miniaturise transistors and double the number of them that could be squeezed on a computer chip roughly every two years, a trend referred to as Moore’s law. The second was an observation from computer scientist Robert H. Dennard that the power consumption of individual transistors fell in proportion to their reduced size. This meant you could double a chip’s computing power at the same time as its energy efficiency.

Breaking Moore’s law

However, these advances couldn’t last forever. Roughly 15 years ago, silicon transistors started getting so small that further efficiency gains became physically impossible. Today, the number of transistors per chip continues to rise, but packing them closer together is getting increasingly complicated. This is because the extra heat given off causes chips to malfunction. “We really are hitting a wall in terms of designing these at smaller and smaller dimensions,” says Iris Bahar at Brown University in Rhode Island. “We are getting close to the limit of how efficient they can be.” For the first time in 50 years, Moore’s law is beginning to falter (see “Power slump”).

At the same time, demand for ever smaller, ever more powerful computers is booming. Upgrading mobile data networks from 4G to 5G technology, which will make download speeds up to 100 times faster, is expected to see average monthly data use in North America balloon from 8.6 gigabytes per person in 2019 to 50 gigabytes in 2024. There has also been an explosion in web-connected devices, from smart fridges and fitness watches to smart meters and factory monitoring equipment, all pumping out data. This so-called internet of things spanned 9.5 billion devices in 2019 and is due to reach 28 billion by 2025.

In 2017, Huawei researcher Anders Andrae predicted that the tsunami of data crunching resulting from all this would consume a fifth of the world’s electricity by 2025. That figure understandably made headlines. His latest predictions are less apocalyptic, but still suggest that more than 10 per cent of the world’s electricity could be devoted to information processing by 2030. In terms of raw power, that would be more than is currently used by the whole of the EU. Such growth is likely to represent a significant future source of carbon emissions, and keeping data centres from overheating will require unsustainably vast quantities of cooling water.

“We might be able to design computers that are three or four orders of magnitude more energy-efficient”

Not everyone believes such a dramatic energy crunch is coming. “Andrae’s models are pretty simplistic frankly,” says Eric Masanet at Northwestern University in Illinois. He says they extrapolate from older studies of computing’s energy use, an approach that has historically led to overestimates. In a recent study, Masanet and his colleagues found that the energy use of data centres only increased by 6 per cent between 2010 and 2018 despite a 550 per cent increase in their workload.

That was thanks to improvements in hardware as well as energy management, but Masanet thinks further efficiency gains will be required. “We need to start paying more attention to the potential for rapid growth in energy use and we need to start doing what we can to avert that,” he says.

Turning down the heat

That’s precisely the motivation behind a host of alternative technologies hoping to exploit new materials or innovative means of manipulating data (see “When the chips are down”). But experts agree it will be many years before any provide sufficient improvement to head off the problems faced by computing.

A more radical approach may now be emerging. A growing number of researchers are revisiting Landauer’s calculations and using new tools to dramatically expand our understanding of how thermodynamics and computing interact. For Jim Crutchfield at the University of California, Davis, computing’s situation has parallels with the industrial revolution. In the 18th and 19th centuries, engines and pumps were built to convert heat into mechanical energy long before physicists formalised the principles of thermodynamics. When those principles were better understood, the gains in efficiency and power were astonishing.

“We’re in the information age and we’re kind of in the same conceptual situation,” says Crutchfield. “The general claim is that if we understand these trade-offs, we’re going to be able to design computers that are three or four orders of magnitude more energy-efficient.”

The opportunity arises from what Landauer’s work left out. For all its profundity, says David Wolpert at the Santa Fe Institute in New Mexico, it could only establish a maximum efficiency, not set out a road map to getting there. In part, that was due to the limited mathematical tools available at the time. They could only describe systems in equilibrium, where no energy enters or leaves, which is a massive oversimplification. “Computational systems are, if nothing else, extraordinarily non-equilibrium,” says Wolpert. They are in constant flux, and information and energy flow in and out of them all the time. In fact, almost nothing in the real world is in equilibrium, he says, something early computer pioneers were aware of but lacked the equations to describe.

There’s another, equally important, property of equilibrium that the real world breaks. According to the laws of thermodynamics, an equilibrium system is in a state of perfect internal disorder – it has maximum entropy. But this is obviously not the case in everyday life, where the entropy of a system is constantly changing. Imagine pouring a jar of marbles down the stairs, for example. You know that all the marbles will eventually reach the bottom, but one or two might bounce back up some stairs on their way down. Averaged over the whole jar, this effect is insignificant, but on small distances and on short timescales, reversals of disorder are possible. The same dynamics play out at the nanoscale, a complexity that the traditional theory of thermodynamics couldn’t predict or describe.

“28 billion
the number of devices projected to be connected to the internet of things by 2025”

This began to change in the late 1990s, when physicist Christopher Jarzynski and chemist Gavin Crooks developed equations that allow us to precisely predict when such entropy reversals happen and how much energy non-equilibrium processes really use. These findings were groundbreaking, says Wolpert.

In recent years, there has been a growing realisation that this field could also help revolutionise computing, for example by charting the way to redesigning hardware as well as software to be more energy efficient using thermodynamic principles. “It’s extraordinarily powerful,” says Wolpert. “These tools they’ve developed can actually be used to go back and expand every single chapter in the computer science textbook.” New equations built off the back of Crooks and Jarzynski’s work can precisely calculate the energy required for a host of information processing tasks in much more realistic scenarios. Although Landauer’s limit remains far off, that could open the door to enormous efficiency gains.

These advances are uncovering new layers of complexity, says Wolpert. Minimising the energy your circuit uses requires complex trade-offs between things like speed, accuracy and physical layout. Wolpert has developed equations that can precisely calculate the thermodynamic cost of different circuit designs. While the ideas are still at an early stage, they could allow us to build significantly more efficient circuits from the ground up. “This is taking the exact same devices and simply wiring them a different way,” he says.

How far these ideas could take us is unclear, but proponents frequently point out that nature has already created a supercomputer that runs on a third of the energy your laptop uses – the human brain. This demonstrates that, at least in principle, new horizons are there to be unlocked. How nature achieves such efficiencies within the brain remains a mystery, but biology offers plenty of other examples of ultra-efficient information processing. Cells rely on a cascade of reactions to process chemical signals from the outside world, operating only about 50 times above Landauer’s limit. The process by which enzymes copy DNA is similarly efficient, leading Microsoft to start investigating it as a potential computing technology.

“The most exciting perspective is whether living systems compute in ways we are absolutely not aware of,” says Massimiliano Esposito at the University of Luxembourg. From what we know so far, that seems likely. Today’s computers march to an internal clock, churning through sequential tasks to produce consistent outputs. That predictability is hard-won. On the tiny scale of transistors, everything from overheating to manufacturing defects can throw computations off. Engineers deal with this by building in wide margins of error and plenty of redundancy, but that, in turn, increases the energy required per bit.

Wasting energy would be a major disadvantage for living systems, says Esposito, and most information processing in biology appears to maximise efficiency at the expense of other attributes like accuracy or speed. These are exactly the kinds of trade-offs non-equilibrium thermodynamics predicts.

As an example, most computers are governed by a single central processing unit that controls all other components. This centralisation is powerful, but means billions of instructions need to be fired off every second with incredible accuracy, at significant energy cost. By contrast, nature gives every cell in the body the power to independently implement the instructions contained in DNA, allowing them to perform in unison without the need for a conductor. That means that a process like DNA copying is able to trade speed and reliability for far greater efficiency. If we want to mimic nature’s extreme efficiency, we may have to borrow some of these ideas.

Computing 2.0

What such a dramatic paradigm shift would look like in computing is hard to imagine, admits Crutchfield, but it holds vast promise. Despite its rapid growth, the internet of things is hamstrung by the need to power billions of small devices, often in locations that make them hard to recharge. In these circumstances, being able to mimic the slower – but ultra-low power – distributed computing seen in nature could be very attractive.

The hardware of the future is unlikely to stray far from today’s chips and circuit boards, says Crutchfield, as that is the only approach we know how to scale. However, a hybrid approach combining traditional computing logic with components that harness thermodynamic effects is a likely starting point. The cryogenically cooled superconducting circuits used in quantum computers hold promise too, says Crutchfield, as they can also operate under conditions where non-equilibrium physics dominates. He has recently teamed up with applied physicists to test some of the field’s predictions on these circuits, and says they could conceivably be scaled up for more sophisticated computing.

Not everyone is sold. The most glaring problem is that most work so far has dealt with theoretical circuits processing just a few bits of information. On the scale of today’s silicon chips, though, these thermodynamic benefits would pale into insignificance compared with the waste energy they produce. “I’m all in favour of theory, but sometimes it just ignores reality,” says Eli Yablonovitch at the University of California, Berkeley.

“Being able to mimic the slower, but ultra-low power computing seen in nature could be very attractive”

Bridging the divide between theory and engineering will take a lot of work from both sides, says Stephanie Forrest of Arizona State University. For one thing, the new mathematical tools that work on the level of bits rapidly become intractable once you start scaling them up. She believes that significant mathematical shortcuts will be needed before real-world computers can start benefiting from these breakthroughs.

Crutchfield agrees. That is why his research programme for the next decade will be dominated by putting the new ideas to the test. All the same, he wouldn’t be surprised to see significant progress before then. “Maybe in about five years, people are going to appreciate what a huge revolution this has been,” he says.

There are signs people are starting to appreciate it now. X, the so-called moonshot arm of tech giant Alphabet, recently hired Crooks to look into the potential applications of non-equilibrium thermodynamics. There are serious practical difficulties to putting these ideas to work, he says, but he doesn’t see any fundamental barriers.

“It’s not just a question of an incremental change in the energy dissipation of the things we’ve got now,” says Crooks. “It would enable entirely new things.” In a world awash with data, that could be just the lifeline we need.