91av

Not so long ago the seaside speak-your-weight machine was about as sophisticated, and irritating, as a talking computer could get. Today, though, the world is filling up with disembodied voices keen to send you to the right cashier, encourage you to listen carefully to the following options, or convince you that your call is important to them whilst leaving you hanging on the line. Even cars think they know the way better than we do.

No one likes being bossed around by a machine, but there may be more to our annoyance at artificial voices than first appears. Our problems with synthesised voices, it seems, stem from the way we are hard-wired to respond to real ones – we don’t like artificial ones because we don’t like their social skills. Now researchers are trying to design artificial voices that don’t drive us to distraction; voices that are not only as good as the real thing, but better.

“Our problems with artificial voices stem from the way we respond to real ones”

Designers of automated call centre systems and in-car navigators are watching with interest. If voices could be designed to be not only friendlier, but also more calming and persuasive, it would pave the way for armies of virtual salespeople and customer servive representatives. Cars could offer traffic advice and service updates alongside directions. Artificial voices would be everywhere, but we would no longer mind. Or that’s the idea.

Scientists have long been interested in creating a natural-sounding artificial voice. As early as the 1780s, the Hungarian inventor Wolfgang von Kempelen succeeded in developing a “speaking machine” capable of producing words and short sentences. In the late 19th century, as a teenager, Alexander Graham Bell even trained his dog to growl continuously while he manipulated its larynx, in an attempt to shape the sounds into words. By the 20th century it was possible to recreate the sound of the human voice electronically. In 1939 Homer Dudley of Bell Labs unveiled the , short for voice encoder – a machine that converted a tone and white noise into speech using a set of mechanical controls that altered rhythm, pitch and inflection, and was “played” like a musical instrument. It was not until the digital revolution of the 1970s, though, that speech synthesis came into its own.

In 1978 Texas instruments broke new ground with their “Speak’n’Spell” toy, which converted words typed on its keyboard into speech using software driven by mathematical models of sound moving through the human vocal tract.

The following year, what we now know as the digital voice of Stephen Hawking was created in the speech lab at the Massachusetts Institute of Technology. Speech synthesis pioneer Dennis Klatt called his brainchild “Perfect Paul”, later following it with another, more gravelly male voice, “Huge Harry”. This, like the Speak’n’Spell, was based on turning written text into sound using software programmed with the rules of language – taking parameters such as frequency, pitch and volume and turning it into artificial speech.

In the early 1990s speech synthesis researchers abandoned attempts to create human sounds from scratch and instead began using real voice recordings. This so-called “concatenative synthesis” takes real speech, cuts it into its component sounds, or phonemes, and puts them back together again to form new words and sentences. While storing a copy of every word in a language would overload the average computer, handling a far smaller number of units in this way makes the task manageable. .

Today’s talking computers, including the voices that read weather reports for the US National Oceanic and Atmospheric Administration system and British Telecom’s SMS text message readers are more sophisticated, using not just phonemes, but diphones – the sound of the transition between two phonemes – and demiphones – one half of a phoneme. This makes them sound much more human than before.

Despite this progress, even the best artificial voice systems remain devoid of convincing emotion. And this, says Clifford Nass, professor of communication at Stanford University in California and co-author of the book Wired for Speech, is why we find them so annoying.

“Over 200,000 years of evolution speech has been an important part of the human environment,” he says. “It was inevitable that our brains would become acutely sensitive not only to the words spoken, but to how those words were said.”

Nass has been studying human-computer interactions for the past 15 years. He says that because we are hard-wired to interpret every voice as if it were human, even when we know it comes from a computer we still can’t help making the same kinds of assumptions about personality, trustworthiness and intelligence. If the “person” sounds bored, insincere or condescending, we react accordingly.

In a series of experiments between 2000 and 2005, Nass and his colleagues set out to discover if changing the qualities of artificial voices could make them more likeable and believable. By altering pitch, speed, volume and intonation, they could mimic different kinds of voices and study their psychological effects.

In one study they changed the artificial voice’s perceived gender. Could making a voice sound more male or more female affect peoples’ decisions? Nass and his team presented volunteers with an ethical dilemma and offered them advice via either a male or female-sounding computer voice. They found that volunteers were more likely to conform to the advice of a voice whose gender matched their own.

The team also found that if an artificial voice is to be trusted as a salesperson, its “personality” can be more important than what it actually says. The researchers created an online auction site and invited two groups of people earlier identified as either extroverts or introverts to bid for antiques. There were two sales pitches for each item: the “extrovert” one was longer, more colourful and enthusiastic, and peppered with comments like “I’m sure you will like this”. The “introvert” sales pitch was shorter and confined itself to the facts. They found that, no matter which description it delivered, extrovert people were more likely to be swayed by an extrovert synthetic voice (wav. format, 1.3 MB) and the introverts by an introverted voice (wav. format, 1.5 MB). (Journal of Experimental Psychology: Applied, vol 7, p 171.)

Then Nass and his colleagues made an alarming discovery: your response to an in-car digital voice could be a matter of life and death. They asked volunteers to watch either a happy or an upsetting film before “driving” around a virtual track following instructions from a computer voice. The type of voice either matched or clashed with the mood of the film and hence the drivers’ state of mind. The results were shocking. Drivers given a voice that matched their emotional state had fewer than half as many virtual car crashes as those with mismatched voices. In fact, it offered a bigger safety improvement than virtually any other car-based change. Perhaps the current fad for personalising sat nav systems with comedy celebrity voices might not be the safest way to get your directions (see “Once is funny…” below).

A similar situation seems to apply outside the lab, as German car manufacturer BMW found to its cost. In 2001 the company had launched a state-of-the-art satellite navigation system in its high-end 7 Series range. The system was the most accurate the company had ever designed, but drivers reacted so badly that BMW recalled the product. Apparently the – mostly male – drivers “felt uncomfortable with, and untrusting of, a female giving directions”, says Nass, who was drafted in to help revamp the system.

“Male drivers felt uncomfortable with, and untrusting of, a female giving directions”

So BMW went in search of the perfect in-car companion. Market research offered a profile of the 7 Series’s typical driver and brainstorming sessions followed to suggest or rule out what kinds of voices they’d most like to hear. Unsurprisingly “mother-in-law in the back seat” was an early casualty, but so was “golfing buddy”, because people don’t like being bossed around by their peers, says Nass. After tests with characters as diverse as “German engineer” to “friendly cowboy” they settled on a male “co-pilot” – highly knowledgeable on all technical aspects of the vehicle (as it had to be the voice of the in-car warning systems as well as the navigator) but always deferring to the judgement of the driver. Its voice was slightly dominant, fairly friendly and very confident. The pitch was relatively deep and it spoke a bit faster than average. To reinforce its subordination to the driver, BMW’s upgraded talking navigator never presumed to say “I”. This time there was no driver revolt.

Calming computers

When it comes to artificial voices, time and again it is similarity – matching the voice to the user – that seems to be a recipe for success.

In practice, though, any system able to do this will need to be able to detect human moods, and select a voice that is the closest possible match. So far, research into mood detection software has homed in on detecting anger and stress – the two emotions most relevant to keeping people safe on the roads, and customers on-side.

“Someone who’s stressed will tighten their vocal cords and speak in a slightly rougher tone of voice,” says John Hansen of the University of Texas at Dallas. “And whereas the exact profile of those changes varies from person to person, there are stress characteristics common to all speakers.”

Anger is slightly more difficult to detect. Felix Burkhardt and his team at T-Systems, a company providing communications systems for businesses, have been programming computers to detect anger in the voices of callers to an automated customer helpline. The biggest challenge, says Burkhardt, is that we all get angry in different ways – some shout and swear, others are cold and sarcastic. On top of this, people tend to adopt a stilted, bullying approach to digital assistants. “People are not very polite to robots,” he says. And by the time someone calls the helpline, they may already be annoyed, the more so by having to speak to a machine. This makes it hard for researchers to get a baseline of normal speech for each irate caller.

Burkhardt’s system uses a two-pronged approach, with a swear-word spotter working alongside speech-analysis software that measures key aspects of the caller’s voice, such as rhythm and stresses, pitch and volume, to come up with a measure of their likely level of frustration. Like Nass, Burkhardt found that we react to computers in a human way. “People used female-oriented swear words in response to a ‘female’ computer voice,” he says. “It’s as if they didn’t realise they were talking to a machine.”

However, using a computer to detect expletives proved to be a non-starter. People can be very imaginative when it comes to swearing, and too often callers used swear words that the system was unfamiliar with. In fact, 99 per cent of the words the system thought were expletives were false alarms.

The speech analysis tool fared better, focusing on acoustic features in the caller’s voice, to work out the statistical probability that the caller was angry. Try saying “How dare you speak to me like that!” and the qualities of angry speech are immediately apparent.

The machine got it right 70 per cent of the time. Not bad, but not good enough to pick out the angry customers and forward them to someone who could calm them down. What we really want to do, says Burkhardt, is to mimic the human ability to recognise anger. Until the accuracy of anger-detection systems reaches 98 per cent, he says, it’s best to leave the emotion detection to humans.

Even if computers could be developed that respond to our emotional state and adjust their demeanour accordingly, not everyone is convinced that this would stop them rubbing us up the wrong way. “Faithful mimicry of human speech, while helpful, is not sufficient to overcome the annoyance of [this kind of] service,” says Ben Shneiderman, a computer-human interactions specialist at the University of Maryland. And even if it could, Jaron Lanier, of the Center for Entrepreneureship and Technology at the University of California, Berkeley is unconvinced that it would be a good thing. In fact he is uncomfortable with the whole idea of computers being used to manipulate our emotions. “My main objection to getting people to accept a computer as if it were a person is that it sets up a vicious cycle; the real person is likely to change her standards to accept the fake person – and thus makes herself into an idiot to make the machine look smart,” he says.

Nass, however, believes that a world where computers can understand our moods and respond accordingly, will be a vast improvement on the emotionless robots we have now. One day, he believes, we may no longer dread speaking to a digital assistant, but be glad that we haven’t been put through to an inefficient, moody, imperfect human. Whoever is right, one thing is inevitable; the robots are coming to a telephone near you.