Technicians test the robots before a robot football match during the 2008 Robot World Cup Soccer Games (Robocup)
ALAN TURING’S computing skills may have helped defeat Nazi Germany but the test that bears his name is virtually useless and needs to be replaced with something better.
So says Paul Cohen, a leading researcher in artificial intelligence at the University of Massachusetts at Amherst. Speaking at the American Association of Artificial Intelligence (AAAI) conference in San Jose, California, he said that the test is so hard that it puts researchers off even trying to create a machine with human-level intelligence.
Advertisement
Turing proposed the test in 1950 as a way of answering the question: can machines think? In the test a human judge chats to a machine and another human. If the judge is unable to reliably distinguish which of them is the machine, the machine can reasonably be called intelligent.
There is nothing wrong with having difficult tests, Cohen says, but the Turing test is not a particularly good one. It does nothing to help researchers improve their programs. “It’s not so much a test as a goal,” he says. If you fail the test you don’t know why you failed, so it’s almost impossible to improve. And if you pass you don’t know why you passed, so you are no closer to understanding your goal.
The Loebner prize claims to be the first practical implementation of the Turing test. Every year people enter with chatterbot programs that attempt to convince judges they are human (see Panel). But Cohen rejects these efforts. “The programs are terrible. They are more about fooling people than creating artificial intelligence,” he says.
But Ron Chrisley an AI expert at the University of Sussex in the UK, disagrees. The contestants in the Loebner price use a series of tricks, such as matching common questions with common replies, that could eventually be used to pass the Turing test, he says. But this merely highlights the problems of the 54-year-old test, which Chrisley agrees is showing its age. “I certainly think we need something else,” he says.
Cohen proposes a series of more realistic and attainable challenges to act as stepping stones, serving almost as qualifying trials for the Turing test. He points to robot soccer as a good example of where to start. RoboCup is a competition that pits robots against human players, and the organisers’ aim is to see a fully autonomous robot team beating a human side by 2050.
RoboCup has a series of competitions that are intentionally easy. Cohen says the rules of the competition are tightened up each year to make it slightly harder, to encourage programmers to produce better programs. “It’s a very powerful idea. People were on the edge of their seats,” says Ron Brachman, president of the AAAI.
