LIKE other human champions facing a machine opponent, Grzegorz “MaNa” Komincz rated his chances. “A realistic goal would be 4-1 in ,” he told an interviewer before the match.
Advertisement
One of the world’s best players of video game , Komincz was at the height of a successful esports career. Artificial intelligence company DeepMind invited him to face its latest AI, a StarCraft II-playing bot called AlphaStar, on 19 December 2018.
Komincz was expected to be a tough opponent. He wasn’t. After being thrashed 5-0, he was less cocky. “I wasn’t expecting the AI to be that good,” he said. “I felt like I was learning something.”
It was just the latest in a series of unexpected victories for machines that stretch back to chess champion Garry 貹DZ’s 1997 defeat by IBM’s Deep Blue. In 2017, another of DeepMind’s AIs, AlphaGo Master, beat the world number one Go player a decade before most researchers predicted it would be possible. The company’s AIs then mastered chess and StarCraft – a game played with dozens of different pieces with hundreds of moves a minute.
But this isn’t just a case of humans being humbled by superhuman AI. The real story is that each win gives us a glimpse of how AIs will make us superhuman too. That’s because thinking is set to become a double act. Working together, humans and AIs will bounce ideas back and forth, each guiding the other to better solutions than would be possible alone.
The potential goes far beyond games. The hope is that this teamwork will help us make vital breakthroughs in energy use, healthcare and more.
This is a vision promoted by DeepMindco-founder Demis Hassabis. Many others agree. “It will be an amazing extension of thought,” says Anders Sandberg from the Future of Humanity Institute at the University of Oxford.
Komincz felt his defeat was instructive. Another StarCraft II professional, Dario “TLO” Wünsch, also beaten 5-0, felt the same. “AlphaStar takes well-known strategies and turns them on their head,” said Wünsch. “There may still be new ways of playing the game that we haven’t fully explored yet.”
Their comments echo those of a growing number of defeated humans. Many are startled by the ability of DeepMind’s AIs to make winning moves no human player would dream up, rewriting centuries-old playbooks. Tapping into these AIs can take players to a new level. After losing to AlphaGo, European Go champion Fan Hui trained against the AI and boosted his global ranking from 600 to 300 in just a few months.
“Tapping into world-beating AIs can take human players to a new level”
Computers have been far better than people at chess for decades. For most players, that was true even before 貹DZ’s historic loss to Deep Blue hammered the point home. All professional players now practice with chess computers. These tend to play defensive games, so the style of top players has become more defensive too.
A further development of DeepMind’s game-playing AIs, AlphaZero, has shaken up the chess world again. In a series of games in 2017 and 2018, AlphaZero beat Stockfish, one of the best chess computers in the world. Unlike Stockfish, AlphaZero plays an aggressive game, often sacrificing pieces early on if this helps it achieve its goals. “AlphaZero just goes for the attack straight away,” says Natasha Regan, who has represented the UK at both Go and chess.
Regan and grandmaster Matthew Sadler have co-authored a book called , which explores AlphaZero’s groundbreaking chess strategies and offers advice for would-be challengers. The AI is more like a maverick human player than a typical chess computer, they say, which makes it a more fascinating tutor.
Creative aggression may be a common trait. Good StarCraft players usually build defences in the early stage of a game. But Regan and Sadler noticed that AlphaStar didn’t bother. They recognised some of the same tactics used by AlphaZero in chess.
“It really struck a chord,” says Sadler. “You really do start thinking there’s an AI style shared across these different challenges.”
The result is a new kind of software that displays what looks very much like creativity and – whisper it – intuition. David Silver at DeepMind is also struck by these thoughts. “The professional Go players who competed with AlphaGo repeatedly remarked on the creativity of the system,” he says. “They expected it to play in a way that was perhaps dull but efficient and instead there was real beauty and inventiveness to the games.”
“It makes sense that we should solve problems in tandem with machines”
So why do these AIs surprise us more than earlier software? The most likely reason is their lack of human bias. As good as previous chess computers are, they have human strategies built in. DeepMind’s AIs learn by playing against themselves. Their algorithms may be different, but their general approach is the same.
All use a machine-learning technique called deep reinforcement learning. This boils down to building a neural network – software loosely modelled on the brain and capable of performing a particular task – by training it on large amounts of data. In a process of trial and error, successes, such as winning a game of Go, are rewarded, reinforcing a particular behaviour.
AlphaGo and AlphaStar learned by themselves, following human examples. But AlphaZero uses only the rules of the game – the “zero” stands for zero input. Instead, it is given the rules and a goal, then left to its own devices. Starting randomly, it plays itself over and over again until it figures it all out. On the way, it picks up its own method of doing things. In just a few hours, AlphaZero played itself tens of millions of times, to become the best Go player and then the best chess player ever. “AlphaZero discovers thousands of concepts that lead to winning more games,” says Silver. “To begin with, these steps are quite elementary, but eventually this same process can discover knowledge that is surprising even to top human players.”
Silver and his colleagues focused on games because they are excellent test beds, offering a wide range of challenges that are familiar to humans. But the end goal of AI development is far more ambitious. “In terms of what’s next, we think our approaches could be applicable to some fundamental problems in science,” says Silver.
An early glimpse of what might be possible came last year with , a DeepMind AI that predicts the intricate structures of proteins. A better understanding of how proteins work will help us control everything from disease to food production. But a protein’s function is determined by its unique structure. And that structure, which usually looks a bit like a tangled rope, is hard to predict from the sequence of its constituent amino acids. Researchers rely on laborious, expensive structure-determination methods that don’t work for many proteins. Cracking how a protein folds based on its amino acid sequence is a very desirable goal, but despite people pursuing this for 70-odd years, it is still largely elusive.
In July 2018, AlphaFold won the Critical Assessment of Protein Structure Prediction challenge, the gold standard for assessing software that aims to predict how proteins fold. The hope is that AlphaFold will bring to future efforts to predict protein structure what related AIs bring to games. So where do we go from here? How far are we from realising bigger goals?
“Sure, we’ve made great progress but I don’t think anybody really thinks they’re inches away from the human level,” says Ken Stanley at the University of Central Florida, who is the founder of the AI lab at Uber. Although AlphaZero used the same algorithm to teach itself to play Go and chess, its chess-playing neural net can’t play Go and its Go-playing one can’t play chess. AlphaZero isn’t able to apply lessons learned in one game to another.
For deep learning to reach the next level on the long climb to human-like intelligence, neural networks need to become generalists. These may involve wholly new kinds of interconnectivity or new rules for activating the software neurons of the network. “We don’t just want complexity for its own sake,” says Stanley. “We want it because it produces things that are amazing.”
Stanley hopes to reach generalisation through neuroevolution: neural networks that improve themselves using techniques inspired by nature. The basic way of using evolutionary techniques in computing is to start with random solutions, select the best ones, mix them together and try again. Repeat this millions of times and the system will converge on a successful AI design on its own.
Assuming such techniques work and we can build ever better AIs, the most promising possibility is that they will become our collaborators. “It is inevitable that humans will be interacting with AI to accomplish tasks,” says Devi Parikh at the Georgia Institute of Technology in Atlanta.
My other brain is a computer
For successful collaboration, we require trust, which for Parikh means we need to develop an AI theory of mind. In humans and some other animals, a theory of mind lets individuals ascribe mental states to others. It lets someone see another’s point of view, their beliefs and intentions.
The more aware people are of what is going on in a colleague’s head, the more effective they are at working together and adapting to each other’s strengths and weaknesses, says Parikh. Why shouldn’t the same apply to teams made up of humans and AIs? (See “Alien thinking).
Part of this will come from making AIs more interpretable and transparent. But the most fruitful collaborations may arise from partnering up and thinking alongside them. Everyone knows that Kasparov lost to Deep Blue. What is less well known is that Kasparov then went on to invent advanced chess, also known as centaur chess, in which humans and computers team up and play in pairs.
As expected, amateur players with computers can beat grandmasters playing alone. But even when both use computers, amateur players can beat grandmasters. When this happens, it is typically because the amateur is a better team player than the expert, who is more likely to disregard the machine’s suggestions.
Of course, there are situations where we should expect human judgement to beat an AI’s. Matters of aesthetics or ethics are the most obvious candidates. Take generative design, where AIs are already being used to create thousands of blueprints for potential parts of aircraft and cars. The program whittles down many potential options to a few strong candidates that a human then chooses from. There are good reasons to keep humans in the loop with autonomous weapons systems, too.
If we can work out when an AI should ask a human for help, the combined thinking could be far more powerful than that of the AI alone. For Sandberg, the trick would be to have an AI that does a lot of our thinking for us but which only makes decisions that we are comfortable with. Owain Evans, a colleague of Sandberg’s at the University of Oxford, is trying to teach an AI about human values. By asking questions about potential decisions, the system is learning what it should do in different moral situations. AIs might check in with our values every now and then. Would you disapprove of this action? Would this outcome be OK? “It might be smarter than me, but it’s still only doing things that I would do,” says Sandberg.
Stanley agrees about the value of human input. He says he was blown away by experiments he conducted with a neural network evolved to guide a robot through a maze. With occasional human input, the AI got much better. It makes sense that we should begin to solve problems in tandem with machines. We can’t beat them at individual tasks, so let’s join them.
“If we play things right, we are going to broaden the way we can think about problems,” says Sandberg. “We know that problem-solving is helped by having different perspectives. Soon, we could have perspectives that are different from any we’ve had before.”
Alien thinking
The ability of AIs to think outside the usual boxes could provide the breakthroughs we need in tackling some of the world’s biggest problems. Yet will we be happy with what the machines come up with?
Sometimes, an AI’s solution to a puzzle is no help, but even when it makes sense, we may feel uncomfortable. For technical problems such as curbing energy use or designing chemical reactions, people will probably go along with an AI. But when it comes to social problems, it might be hard to shrug off the feeling that we know better.
For example, imagine that instead of voting for who we want in a government, we asked an AI to assess the strengths of the various candidates and pick for us. If its choices didn’t fit our expectations or preferences, would we go along with them?
It might be the same for moral issues. “If an AI that was always right about stuff started giving me moral advice, I might think twice about following it even if intellectually I know I ought to,” says Anders Sandberg at the University of Oxford. “I might just want to decide myself.” Or maybe not – it would be fascinating, if somewhat dystopian, if people absolved themselves of decision-making.
In principle, testing ideas to figure out what works best and then basing policy on those results makes lots of sense. But Sandberg thinks this approach won’t work for issues that evoke strong feelings in us – how our children are taught, for example. This is why attempts to run policy trials in schools have proved controversial.
It is likely to be even harder to accept an AI’s recommendations in such circumstances, especially if they seem strange. “AI will probably be able to tell us how to educate children, but will we want it to?” asks Sandberg.
