91av

QUANTUM mechanics is often called a theory of the very small. In reality, it explains phenomena on a vast range of scales – from elementary particles and their interactions, through atoms and molecules, all the way to neutron stars and the supernovae that spawn them. So far, essentially all its predictions have been confirmed by experiments. It is the most successful theory of material reality we have ever had.

So why have so many physicists, from Albert Einstein onwards, taken the view that quantum theory is wrong?

The reasons lie in its mysterious nature, in the phenomena it doesn’t explain and the answers it doesn’t give. That is reason enough to seek what might lie beyond it. I believe we already have the outline of what this deeper answer looks like. We are only at the start of this work, but by digging down into the fundamental principles that underlie reality, and weeding out what is right and what is wrong about our current ideas, we can see glimpses of a truly unifying picture of physics. It comes at a price: to go beyond quantum, we must totally upend long-held ideas of how the universe hangs together.

It is easy to state the basic problem of quantum mechanics as a theory of reality: it doesn’t tell us what is happening in reality. It has two different laws to describe how things and events evolve. The first applies most of the time, and describes quantum objects as wave-like entities embodied in a mathematical construction known as a wave function. These objects evolve smoothly in time, exploring alternative realities in “superpositions” in which they aren’t restricted to being in any one place at any one time. That, to any intuitive understanding of how the world works, is distinctly odd.

Curiouser and curiouser

The second law applies only under special circumstances called measurements, in which a quantum object interacts with a much larger, macroscopic system – you or me observing it, for example. This law says that a single measurement outcome manifests itself. The alternative realities that the wave function says existed up to that point suddenly dissolve.

These two laws exist in parallel, in apparent contradiction of one another – a fundamental failure of our understanding known as the measurement problem. Attempts to do the obvious, and derive the second law from the first, have so far failed. We are left with only statistical predictions of what is going on in the quantum world before it is measured.

The mysteries don’t end there. Quantum theory also seems to violate the principle of locality, which says that objects or events must be near one another to interact. In classical physics, for example, the gravitational or electrical force between two objects depends on their distance: the closer they are in space, the stronger the force between them. Quantum theory, meanwhile, introduces entanglement, a phenomenon that allows objects to seemingly influence each other instantaneously over any distance.

Einstein notably believed that these blemishes indicated that quantum theory was wrong, and that a truer, deeper description of nature was out there. He wasn’t the only quantum pioneer to express doubts. Louis de Broglie, who first predicted the wave-like aspects of matter, was another sceptic, as was Erwin Schrödinger, whose famous thought experiment of the dead-and-alive cat was designed to highlight the absurdity of quantum theory’s prediction of alternative realities. In the present day, quantum dissidents include notable physicists such as Roger Penrose and the Nobel-prizewinning theorist Gerard ‘t Hooft.

Arguments about whether quantum mechanics is a complete theory of reality have usually been carried out in isolation. But the route to a deeper and truer understanding of nature may lie in connecting the problems of quantum theory with other big, open problems in fundamental physics.

The most obvious one is how to develop a quantum theory of gravity. Gravity is the only one of nature’s four fundamental forces not to have a quantum-mechanical description. It is described by Einstein’s general theory of relativity as an effect resulting from massive objects warping space-time around them.

General relativity and quantum theory seem to be fundamentally incompatible, not least in the way the former describes a smooth, malleable space-time. By contrast, quantum theory suggests that it must at some level come in discrete chunks, or quanta, of space or space-time.

We have at least half a dozen ways to get part of the way across this divide, among them string theory and loop quantum gravity. Indeed, the latter idea gives precise predictions for what the quanta of space-time must look like. But we have no idea whether any of the suggested routes are the right one because none predicts an experimental test we can perform with current technology.

Quantum theory and general relativity clash in other ways, too, notably over the nature of time. Relativity makes it impossible to establish one objective “flow” of time of the sort we perceive, with a past and a future separated by a universally defined now. Quantum theory, meanwhile, characterises time as a metronomic “beat” set somewhere outside the universe. So is our perception of a flowing time real, or an illusion?

Back to basics

There are other deep questions. The quantum descriptions of the other three fundamental forces – electromagnetism and the weak and strong nuclear forces – can be bundled together into the so-called standard model of particle physics. But why do these three forces have such very different strengths within the standard model? Then there is the nature of the dark matter and dark energy that dominate the cosmos on a large scale, but which the standard model doesn’t mention. These questions and others concern how our universe came to be, out of a vast number of seemingly equally probable universes allowed by the laws of physics.

To solve all these issues, we need to wipe the slate clean, go back to the first principles of quantum theory and general relativity, decide which are necessary and which are open to question, and see what new principles we might need. Do that, and an alternative description of physics becomes possible, one that explains things not in terms of objects situated in a pre-existing space, as we do now, but in terms of events and the relationships between them.

This endeavour starts with a few basic hypotheses about the nature of space and time. First, that the history of the universe consists of events and the relationships between them. Second, that time – in the sense of causation, the process by which future events are produced from present events – is fundamental. Third, that time is irreversible: causation can’t go backwards, and once an event has happened, it can’t be made to unhappen. Fourth, that space emerges from this description: events cause other events, creating a network of causal relationships. The geometry of space-time arises as a coarse-grained and approximate description of this network.

A fifth hypothesis is that energy and momentum are fundamental features of the universe, and are conserved in causal processes. These five hypotheses define a class of models called energetic causal set models that my collaborator Marina Cortês of the Royal Observatory in Edinburgh, UK, and I . I have since added a sixth hypothesis, a version of the holographic principle first stated by ‘t Hooft. This says that when two-dimensional surfaces are defined in the emerging geometry of space-time, their area gives the maximum rate by which information can flow through them.

“To solve the problems of physics, we need to decide which of its principles are open to question”

In this picture, every event is distinguished by the information available to it about its causal past. We call this the event’s sky because it functions rather like the sky above us does. The sky – or the horizon of our sight more generally – is a snapshot of what we see at any one instant, a two-dimensional surface formed by photons of different colours, informing us of our relationships with the things around us. Because nothing travels faster than the speed of light, only things within an event’s sky can influence it, so the sky is also a view of its causal past.

Sky’s the limit

This picture allows us to describe how information and energy flow through events as the universe evolves. Ted Jacobson at the University of Maryland in the US and Thanu Padmanabhan at the Inter-University Centre for Astronomy and Astrophysics in Pune, India, have independently shown that the sixth hypothesis, together with the first law of thermodynamics, which governs the amount of useful energy available to a process, can be used to derive the equations of general relativity, and hence gravity.

Their work assumes that space-time is always smooth. By marrying their reasoning with the picture of a prototypical discrete, quantum space-time in our models, we can derive both general relativity and smooth space-time as emerging from a dynamically evolving causal network.

As well as providing the seed of a quantum picture of gravity, this immediately solves the problem of the flow of time in Einstein’s cosmos. In a causally defined universe, the most basic interaction is the creation of an event when two “parent” events come together to make something new happen. At each stage in the construction of a space-time history, the future doesn’t exist. But we can postulate a limit to the number of events any parent event can give birth to. Events that have had their full allotment of progeny cannot have any further direct influence on the future, and are relegated to the past: time flows.

The most exciting prospect, which Cortês and I have been exploring over the past few years, is that quantum theory might also emerge from this picture. That comes from building energetic causal set models to answer the key question of which events interact.

Events differ from one another in that each has a different sky, a different view of its causal past. We can define a measure of how similar two events’ views are, and pick the pair with the most similar views to be the parents of the next event. The idea is that the similarity of views can play the role that distance in space does in conventional classical and relativistic physics. The more similar the views of two events, the more likely they are to interact.

The overall effect of choosing the pair with the most similar views as parents pushes both out of the present and into the past. Removing two very similar views and creating a new view that is a synthesis of both – and hence different from both – has the effect of increasing the total diversity of the views of all events in the universe. A measure of the total diversity of an ensemble of views is a quantity we invented in the late 1980s with Julian Barbour at the University of Oxford. We called it the variety of the system.

All this has intriguing consequences. The views are chosen and evolve precisely so that the total variety evolves to its maximum – and it turns out that this exactly reproduces the dynamics of quantum theory.

You can begin to see how this works. Similarity of views only implies nearness in emergent space-time for large, complex events. If an event has a very simple recent causal past, there may be other simple events with similar pasts that aren’t necessarily nearby in the emergent space-time. Yet by the principle of similarity, they have a high probability of interacting with each other.

Einstein and others since have proposed that quantum wave functions describe collections, or “ensembles”, of systems defined by properties they share, but it has never been clear whether these ensembles truly exist. In this “real ensemble” picture, they do. The continual, brazenly non-local interactions between simple, causally related objects widely distributed in space explain all the probabilities, uncertainties and spooky interactions of quantum physics. They only ever occur between simple systems such as single particles on a microscopic scale because only these can have similar views. Large, complex systems with many degrees of freedom – you, me, Schrödinger’s cat – will have a unique causal past. For us, the closer we are in space or space-time, the more similar our view will be. Proximity matters at the classical scale in a way it doesn’t at a quantum scale.

, my collaborators and I have also shown how to describe an interaction among the members of each ensemble that results in the ensemble’s quantum state evolving in time according to the laws specified in quantum mechanics. That gives a simple and elegant solution to the measurement problem.

There remains the question of what happens with systems of an intermediate size, whose causal pasts aren’t unique, but which might have an intermediate degree of causal relationship with things far away in space. These, I predict, should be described by a tweaked version of quantum physics in which the superposition principle fails to hold exactly. It is possible that experimentalists can construct such systems, and test this prediction, using the tools of quantum information. If we can create sufficiently large and complex entangled states, which would have no or only a few natural copies within the universe, our picture predicts that their evolution in time will deviate from that predicted by quantum mechanics.

More details need to be filled in. This is just a sketch of how we might go beyond today’s quantum picture and construct a unified physics that sidesteps the fundamental problems we currently see ourselves facing, while preserving the best of what we have. No doubt it isn’t correct in every detail, and others may come along with other, entirely different ideas. But the current impasse in physics suggests that it is only through bold ideas that we will move forward.