The Program Director of Physics at Chapman University discusses the evolution of quantum foundations, changing perceptions of the field, and his recent work around quantum agency.
Since the birth of quantum mechanics a century ago, there has been a great debate over what the theory says about the world. While this debate raged white-hot at the start, it quickly fell to the background as physicists focused more on the range and applications of the new theory.
For decades afterwards, the study of the questions at the heart of quantum physics – known as quantum foundations – was actively discouraged within physics, in some cases damaging reputations and careers. But the field has seen a renaissance in recent years, with a small but growing community of physicists turning their attention to the deepest questions about the nature of the quantum world.
Matt Leifer is one of these physicists. A professor at Chapman University in California, he describes himself as “straddling the line between mathematics, philosophy, and theoretical physics.” He recently spoke with FirstPrinciples about quantum foundations and what exactly is going on in the world of the very, very small.
The interview has been condensed and edited for clarity.
FirstPrinciples: What are the core research questions that quantum foundations tries to address?
Matt Leifer: This year, we’re celebrating 100 years of quantum mechanics, so it’s a fairly old theory. Nonetheless, there’s still widespread disagreement about what it actually says about the nature of reality. Everybody pretty much agrees on the predictions – quantum mechanics supplies us with an algorithm that says, “If you set up the equipment in your laboratory this way, this is the probability that you’ll observe different outcomes.” But there’s no consensus about what that means.
Normally, a physical theory gives you a story. It says, “Here are some things that exist, like space and time and particles.” And then it tells you the rules for how they behave. If that theory is true, it usually means that you believe, at least approximately, that those entities are really out there. But quantum theory doesn’t give us a description like that, and it’s very confusing.
Historically, quantum mechanics developed from multiple strands and they all had contradictory viewpoints, and so we ended up with this mess. One of the central questions of quantum foundations research is simply: Assuming the predictions of quantum mechanics are approximately true for our world, what does that mean for what exists out there, and how does it behave?
Now, that may sound like a fairly metaphysical question, and it is, but it turns out that we can actually constrain reality. We have a lot of potential ideas in quantum foundations, and we don’t know which one’s right, but we’ve ruled out a lot by way of theorems, results, and experiments. For one thing, we know it’s going to be very different from the classical world of Newton’s physics.
One of the more remarkable things that’s happened in the past few decades is that we’ve found out that some of these things that we discovered in quantum foundations are actually useful in practical ways, in quantum information and quantum computing. You may think that’s miraculous, but to me, this is the sign of the foundations of a subject being in a good state, because there should be connections to other areas of physics and science.
FP: As you said, it’s been 100 years since we first developed quantum mechanics. Why has it been so hard to figure out what the theory is telling us about what’s in the world?
ML: There are several ways of answering that question. One way is to point out the history, that the theory didn’t really come from one coherent idea. Usually, physical theories change over time. Back in 1700, you have Newtonian mechanics with particles on trajectories. Initially, it doesn’t have the concept of physical fields, like the gravitational field or the electromagnetic field. That’s added later, but it’s added on top of an already coherent theory. The theory starts from the idea of what exists.
But in quantum mechanics, we had various anomalies, places where classical physics gave the wrong answer. Initially, at the start of the 20th century, quantum theory was this ad-hoc collection of rules, which contradicted existing physics. You couldn’t really combine them with the rules of classical physics and get something coherent, because those ad-hoc rules violated some of the rules of classical mechanics.
So, for a good 20 years or so, physicists are going around having to judiciously decide where to apply these quantum rules. You can’t apply them to everything because you don’t have a full theory. Some things have to be treated according to classical mechanics. We’re introducing a theory where we don’t really know what the things we’re describing are, and there’s a certain amount of ambiguity about how to apply it.
In that situation, along comes Werner Heisenberg, 100 years ago. In 1925, he first formulates what’s called matrix mechanics. His attitude was, “Well, we don’t necessarily understand what things are made out of, so let me just try and focus on the observable predictions.”
A few months later, Erwin Schrödinger comes into the picture, and he comes up with a different idea, wave mechanics. He has a picture where everything is this spread-out wave that doesn’t jump up and down, but just evolves continuously.
Both of these approaches, matrix mechanics and wave mechanics, converged in a single, unified theory. They both had starting premises about what exists out there, but neither of them fully aligns with the theory that we end up with.
We were left in a bit of a mess for a long time, although there were always a few people who thought about this stuff. But, for a long time, it was not very fashionable to work on these kinds of ideas.
And to some degree, that was sensible; it wasn’t exactly clear what the ultimate theory was going to be. Some people thought that quantum mechanics would break down before we described high-energy particle physics and things like that. It takes a while before people realize that, “Oh, actually, quantum theory is going to be adequate for all this kind of stuff.” I think it’s legit to put those things to one side until you know that.
But by the 1970s, and certainly by the 1980s, we’re clear on that, and still not many physicists are working on it. People are working on it in philosophy departments – there have been a few major interpretations of quantum mechanics proposed. But it’s still not hugely popular. Even though quantum mechanics is 100 years old, part of the reason there wasn’t progress throughout that time owes to historical and sociological factors, and not the fact that there wasn’t stuff to be done.
FP: How would you characterize the attitude toward quantum foundations within physics now, and how it’s changed over the last couple of decades?
ML: The acceptance doesn’t have to be the entire physics community for the field to thrive. There just has to be enough institutions and enough positions available.
I was always interested in foundations. But when I did my PhD 20 years ago, there were essentially zero jobs in quantum foundations – like one or two maybe. Now, I would say that it’s not hard for somebody to do a PhD in quantum foundations. It’s not hard to get a postdoc position afterwards. It’s hard to get a faculty position, but it’s hard to get a faculty position in every single field of physics. It’s still a small field, but there are definitely more opportunities, and hence more people doing it.
I think there were always students who were interested in this stuff, but they were discouraged previously, and now that’s happening less. It’s not that the entire physics community has suddenly become very pro-foundations. That’s far from the truth. But I think enough has happened that it’s possible to have a thriving field. To some degree, I’m jealous of the students who are coming up now, because some of them don’t even need to think about these issues.
FP: What are you working on now? What are the research questions that interest you the most?
ML: There are three or four main areas. The most wild and speculative one is quantum agency. What I mean by an agent is just something that can be modeled as if it makes decisions. The basic idea is really about how such things interact with quantum mechanics. And what happens if your agent is themselves quantum? Then you ask yourself, “Well, if I was such a quantum entity, how would I describe the outside world?” And the answer is not obvious.
This question is very interesting to me, because it sort of butts up against the limit of what we can do with the quantum formalism today. It offers the possibility that, actually, maybe where we’re at is not the be all and end all. Maybe it’s missing something that could give us clues about how to think about quantum mechanics in a more general way. It’s the most speculative aspect of my work. But to some degree, it’s the thing I’m most excited about, because if we can actually do something like that, it could really change the way that we understand quantum mechanics.
