91av

Poking your boss in the face is something few people would dream of doing, but today researcher Shuichi Nishio puts aside his inhibitions. He raises his index finger and jabs an unsuspecting cheek. “What are you doing? No, no, no,” mutters his target. Undeterred, Nishio keeps up the harassment, moving his finger over to the smooth-skinned chin. Watch the poking in action (2.5MB, wmv format).

Despite appearances, the object of Nishio’s attention is not a person, but a robotic replica of Nishio’s supervisor, Hiroshi Ishiguro. The real Ishiguro watches this surreal scene unfold on video monitors in a nearby room. He has piercing eyes, a frowning countenance and long hair swept across a wide forehead, like a Japanese Beethoven. So does his robot double, which Ishiguro is controlling via the internet. When the roboticist speaks, his twin moves its lips in sync and replicates his stern voice. “Don’t do that,” it says in accented English. Ishiguro clicks a mouse, and the robot sharply turns its head away from Nishio, blinking and looking peeved. Watch a short video of Geminoid and his creator joined in greeting (2.5MB).

The researchers are experimenting with their latest and strangest creation at ATR Intelligent Robotics and Communication Laboratories near Kyoto, Japan. This machine, called Geminoid HI-1, is the first of its kind. Unlike most humanoid robots, such as Honda’s Asimo, the whole point of Geminoid is to look as human as possible. Cast from moulds of Ishiguro’s body, its movements bear a striking resemblance to his gestures and subconscious behaviours. As it sits in the lab, its eyes dart to and fro and air-powered actuators – small pistons and air bladders that expand or contract when air is pumped in or out – shift its posture and make its legs fidget. The HI-1 looks so human-like that passers-by are reluctant to touch it. When Ishiguro sits next to it, one might mistake them for twins.

This is a startling demonstration. It shows that the field has reached the point at which a robot can pass for human in appearance – in limited doses at least. Boosted by recent advances in materials, actuators and control algorithms, the race is on in Japan, South Korea and the US to push the bounds of believable robots and create a perfect android (see “Droids ‘R’ Us”, below). Some hope that these human-like robots will be welcomed more readily than today’s machines as helpers that will eventually be able to care for the growing ageing population and work as companions, particularly in robot-friendly Japan.

Ishiguro, a visiting researcher at ATR Labs and a professor at Osaka University, has a different outlook. His group studies interactions between humans and robots, trying to pinpoint which social cues are important for getting along so that machines can be built with more human-like qualities. Along the way, Ishiguro wants to uncover exactly what it is that makes our appearance and mannerisms… well, human. “My purpose is to know what it is to be human by developing an android,” says Ishiguro. “That is most important and fundamental for me.”

“Android science has led to a new kind of Turing test: whether a robot can pass for human in appearance”

It is hardly a new quest, of course. Androids – robots designed to look human – have populated science fiction for more than a century. From the Robotrix of Fritz Lang’s Metropolis to the replicants of Blade Runner to the Terminator, fictional androids fool our senses and pass for one of us. In the real world, however, robotics and artificial intelligence researchers have mostly ceded androids to the entertainment and theme-park industries. For decades, the world’s most advanced androids were probably the animatronic Pirates of the Caribbean at Disneyland in California.

One reason researchers have shied away from building more sophisticated androids is a theory put forward in 1970 by roboticist Masahiro Mori. He proposed that our feeling of familiarity increases as robots appear more and more human-like, but that our comfort level plummets as slight defects in behaviour and appearance repulse us, as if we are watching a moving corpse. Mori called this effect the “uncanny valley”. The term is widely used by roboticists and has spread to the animation industry, where it describes people’s reaction to increasingly realistic digital characters (91av, 22 July, p 32).

Ishiguro sees the uncanny valley as territory to be explored and then bridged. Growing up in Japan in the 1960s, he loved robots, but it was art that inspired him, not engineering. He wanted to be an oil painter until the realisation that he had mild colour-blindness put an end to that ambition. At university he studied the mathematics behind how computers can be made to recognise objects. He went on to build a robotic guide for the blind, and later, as an up-and-coming roboticist, he designed an interactive wheeled humanoid called Robovie.

Throughout the 1990s, Ishiguro became increasingly interested in giving robots a realistic human form so they could better communicate with people. “Our brain is designed for recognising people, not for recognising computers or objects,” he says. “Therefore I think androids would be an ideal information medium.” Karl MacDorman, an expert on human-machine interaction at Indiana University’s School of Informatics in Indianapolis, also sees a cultural component to Ishiguro’s interest; Japanese society is perceived as being polite, mannerly and accepting of new technology. “Most Japanese would rather ask an android directions than disturb another person,” he suggests.

By 2002, Ishiguro had completed his first android at Osaka University. “Repliee R-1” was a small droid moulded in the image of his then 4-year-old daughter. It could move its eyes, eyelids, mouth and neck, but the movements looked jerky and unnatural, in part because there were no actuators in the body. To boost realism, Ishiguro collaborated with Tokyo-based animatronics-maker Kokoro, which builds robots for the entertainment industry. In 2003, the company provided him with a more lifelike, adult female droid, animated by an air-powered actuator system. Repliee Q1 displayed no facial expressions but could move its upper body in a smoother, more natural way.

That got Ishiguro thinking. Androids, if convincingly human-like, could serve as experimental “controls” to study human behaviour. Their body language can be precisely programmed and reproduced, and subtly altered to test which mannerisms are important for social interactions. Applying these findings to building better robots amounts to a new kind of Turing test: whether an android can pass for human in appearance, rather than intelligence as the original challenge dictated.

With that goal, Ishiguro’s team studied videotapes of people sitting and conversing, watching for subconscious “micro-movements”, the tiny shifts in position people make even when they are sitting still. The researchers then reproduced those movements in Repliee Q1. In one experiment, 20 people were asked to check the colour of a cloth behind a screen that was drawn back for 2 seconds. They were not told beforehand that there was also an android sitting behind the screen, 3 metres away and facing the subject. Once the screen was back in place, the participants were asked whether or not the figure was human. When the android sat entirely still, 70 per cent of the subjects correctly reported that they had seen an android. When Repliee Q1 made micro-movements, however, 70 per cent reported that they had seen a human.

That sounds impressive, but the android still needed refining. In 2005, Ishiguro unveiled an upgraded Repliee at the World Exposition in Aichi, Japan. The robot was modelled after Japanese television broadcaster Ayako Fujii. A network of floor sensors and cameras enabled it to detect the presence of people, while speech software let it engage visitors in short verbal exchanges. The researchers paid special attention to details, using pliable silicone skin and real human hair styled professionally, while 13 air-powered actuators in the robot’s head let it display rudimentary facial expressions. Another 39 actuators in the upper body enabled its arms to move in a more lifelike way than the previous model. The overall effect was so realistic that some visitors mistook it for a real person.

Though the Repliee robots were starting to look more lifelike, Ishiguro felt a bottleneck approaching. The droids’ level of artificial intelligence was too low to allow them to interact with humans in more complex conversations. “We might be able to compensate for this problem maybe within 100 years,” says Ishiguro. So he took a different tack, setting out to build an android that could be more fully controlled by a human, with the aim of using it in experiments to pin down the visual keys to human-like behaviour and appearance.

Building a remote-control android in his own image seemed only natural. With two jobs and a busy conference schedule, the ability to be in two places at once was too good to pass up. Kokoro supplied the same technology used in the Repliee bots, and earlier this year Geminoid HI-1 was born. Its air-powered actuators are fed by tubes that snake out of its bottom and are connected to a refrigerator-sized air compressor hidden behind a screen. The HI-1 has 46 actuators in total, one of which is dedicated to maintaining Ishiguro’s famous frown.

The HI-1, like the previous androids, must remain seated; its legs were only designed to produce micro-movements such as shifting and jiggling rather than standing or walking. Nishio and the lab staff studied Ishiguro’s mannerisms on video and programmed them into HI-1’s stash of behaviours using custom animation software. The android’s appearance was cast from Ishiguro’s face and body using shape-memory plastic and plaster. The “skin” is made from silicone with a layer of piezoelectric film that can detect pressure when bent. At major joints such as the elbow, the silicone does not fold the same way as human skin, so the metal skeleton is left uncovered and HI-1 usually wears long sleeves.

To control the movements of his doppelgänger, Ishiguro sits at a computer manipulating a mouse. First he selects whether the android’s state is listening, speaking or idle, and chooses its mood – angry, disgusted or happy. Each state triggers about 20 autonomous behaviours, such as breathing, blinking and fidgeting. Ishiguro can then choose an “explicit behaviour” such as bowing, nodding or turning to the right or left. He speaks through a microphone and headset, while an infrared camera system tracks his lip movements with the aid of several markers placed around his mouth. In real time, a computer maps the markers’ locations in 3D and generates corresponding lip-motion commands, which are transmitted to the robot’s actuators. A speaker behind the robot broadcasts Ishiguro’s voice.

The effect is striking, and Ishiguro thinks the HI-1 has finally surmounted the challenge of Mori’s uncanny valley. “Our robot is not uncanny any more,” he says. He is not alone in this view. Robotics expert David Hanson, who has recently built realistic android replicas of Albert Einstein’s head and science fiction author Philip K. Dick, agrees. “Ishiguro’s work is brilliant and ground-breaking,” he says. “It is really important.”

Other researchers, however, are sceptical that Ishiguro’s approach will lead to practical applications. “His work is interesting and there is a place for it in the spectrum of humanoid research,” says James Kuffner of Carnegie Mellon University’s Robotics Institute in Pittsburgh, Pennsylvania. “Ishiguro’s robots are currently more like electronic puppets, and unlike ‘serious’ humanoids designed to do useful work.” Some researchers also find Ishiguro’s practice of replicating himself and his daughter “narcissistic and in some ways downright creepy”.

Far from being deterred, Ishiguro says he is looking forward to the day when he can stay home while his android double travels abroad to speak at conferences. “I can avoid long flights that way,” he says. In the meantime, his group has had staff meetings in the lab – with the robot representing its master. Even when he is out of the country and the robot is shut off, interns are wary around it. They think Ishiguro might be watching them.

The researchers are now trying to make the android move more realistically. Its lip motions are slightly out of sync with Ishiguro’s, and they have a slow, unnatural look to them. Ishiguro has discovered that he is adapting his own lip movements by speaking more slowly. As a result he feels “disabled” when he is at the controls, so Ishiguro’s team is adjusting the system that controls the robot’s lips, trying to find a combination of lip and facial motions that better suits the HI-1. “By analysing actual human behaviours precisely, and by examining the effect of adding or subtracting motion elements from HI-1’s behaviour, we can see which motion is contributing for what purpose, and their relative importance in conversation,” says Nishio.

For now, the HI-1 falls short of looking truly human. In fact Ishiguro doubts they will ever create a perfect android. “Not in 100 years,” he says. “Humans are very, very complicated.” Though we may have bridged the uncanny valley, the differences between androids and humans are still obvious upon close inspection. For instance, HI-1’s eyes do not yet blaze like Ishiguro’s, nor do they look alive and wet like real eyes.

Perhaps this suggests something deeper about our pursuit of androids. Nishio recounts an old Japanese tale about an artist who never drew pupils in the eyes of the birds he painted. One day a king asked why, and the painter replied that if he did, the bird would fly away. The king didn’t believe him and forced him to fill in the eyes. The bird came out of the painting and flew away. “This was something that came into my mind when I first saw the HI-1,” says Nishio. “We are still not as good as the painter. There’s still more to go before our android can ‘fly away’ on its own.”