91av

ON THE night of 3 September 2010, 33-year-old Brian Wood was driving along a highway in Washington state. Asleep in the passenger seat was his wife Erin, seven months pregnant with their first child. The couple were on their way from Vancouver, Canada, to spend time at her parents’ vacation home by the picturesque Puget Sound.

Out of nowhere, a Chevy Blazer came hurtling towards them. By the time Wood saw it, it was too late. He braked hard and swerved right to take the brunt of the impact. He died instantly, but his wife and their unborn daughter survived.

We hope it never happens to us, but any driver might find themselves making such a split-second, life-and-death decision. They are part rational, part reflex, and draw on a delicate balance of altruism and self-interest programmed into all of us. .

As our cars edge towards making these decisions for us, cases like this raise profound ethical questions. To drive safely in a human world, autonomous vehicles must learn to think like us – or at least understand how humans think. But how will they learn, and which humans should they try to emulate? “We’re talking about self-driving cars, up to two tonnes of steel and machine that could crash into homes and people,” says ethicist Patrick Lin of California Polytechnic State University in San Luis Obispo. “We definitely can’t leave it up to manufacturers to do what they want.” It’s time to engage low gear – we have a moral mountain to climb.

The most advanced cars of today boast systems to help you cruise down a motorway (Tesla’s Autopilot), creep along in bumper-to-bumper traffic (Mercedes-Benz’s Distronic Plus with Steering Assist) or detect hazards in poor light using thermal imaging (Audi’s Night Vision Assistant). Some cars can even do the dreaded parallel parking (Ford Fusion) or prevent you from rear-ending the vehicle in front (Infiniti Q50). They still can’t take your non-driving grandparent to bingo, pick the children up from school or let you work peacefully in the back – but with the world market for driverless tech , that’s probably only a short ride away (see “Into fifth“).

“When a philosopher says it’s an interesting problem, engineers tend to just shrug”

The ethical challenges raised by driverless cars can often be reduced to the trolley problem, a thought experiment familiar to philosophy students. Imagine a trolley car out of control, and five oblivious people on the track ahead. They will die if you do nothing – or you could flip a switch and divert the car to a different track where it will kill only one person. What should you do? In a similar spirit, should an autonomous vehicle avoid a jaywalker who suddenly steps off the curb, even if it means swinging abruptly into the next lane? If a car that has stopped at an intersection for schoolchildren to cross senses a lorry approaching too fast from behind, should it move out of the way to protect the car’s passengers, or take a hit and save the children? “Many or all of those decisions will have to be programmed into the car,” says .

Millar trained as an engineer before switching to philosophy, and knows this is no run-of-the mill engineering problem. “When a philosopher comes along and says, ‘There’s this interesting new problem you have to deal with’, the car companies and engineers, very understandably, look strangely at that philosopher and kind of shrug their shoulders,” he says.

Answering such “what do we do if…” questions is a two-step process. First, the vehicle needs to be able to accurately detect a hazard; second, it must decide on its response. The first step mainly depends on the efficient collection and processing of data on the whereabouts and speed of surrounding vehicles, pedestrians or other objects. “Cars need sensors that can give them a picture of the world around them,” says , Pennsylvania. These might include video cameras to read traffic lights and road signs, or systems that emit laser or radar pulses and analyse what bounces back.

Google’s self-driving cars have eight sensors, Uber’s driverless taxis 24, and Tesla’s new cars will each have 21, all combining their data into a stream, rather as we integrate what our various senses are telling us. “In robotics, it’s called sensor fusion,” says , who heads .

“Part of the problem with a rules-based approach is that often there are no rules”

Sometimes the hazards are clear, says Nick Reed, leader of the that will trial driverless shuttles along pavements in Greenwich, London, early this year. Part of the route they are testing involves a riverside path. “We definitely don’t want the vehicle swerving left into the Thames,” says Reed.

Moral drivers

Not everything is that obvious. Consider the only death so far linked to driverless technology, last May. It happened because Tesla’s Autopilot system failed to detect that the whiteness ahead wasn’t part of a bright spring sky, but the side of a trailer. A human might have made that mistake too, but sometimes driverless vehicles make a hash of things we master intuitively. “One of the things AutoNOMOS cars have struggled with is someone walking behind a parked bus,” says Rojas. A human mind would expect them to reappear, and supply a pretty accurate estimate of when and where – but for a driverless car, that’s an extrapolation too far.

Even if a sensor system allows an autonomous car to assess its environment perfectly, the second step to driving in a morally informed way – taking the information gathered, assessing relative risks and acting accordingly – remains an obstacle course. “At a basic level, it’s about setting up rules and priorities. For example, avoid all contact with human beings, then animals, then property,” says Lin. “But what happens if the car is faced with running over your foot or swerving into an Apple store and causing millions of dollars in damage?”

Part of the problem with such a rules-based approach is that often there are no rules – at least, no single set that a sensor system based purely on obvious physical cues could hope to implement. For one thing, they can’t compute societal cues we all rely on when driving (see “Read my mind“). For another, the information a video camera or radar echo can supply is limited. “Detecting a bus is one thing, detecting that it is full of schoolchildren is more difficult,” says London.

Technologically, it’s probably doable. A human intervention might program in details of the number and age of the passengers to be broadcast to surrounding vehicles, or sensors inside the bus might autonomously track its weight, including whether a person is sitting in a particular seat, says Lin. But who decides a hierarchy of what lives are worth, and how do we eliminate discrimination and bias in how the cars are programmed?

There’s one way to avoid such thorny moral questions, says Lin: simply ignore them. After all, a human driver is likely to know nothing about those in the vehicles around them. “We could avoid some ethical dilemmas by being deliberately blind to certain facts,” says Lin. This “veil of ignorance” approach amounts to developing responses to simple versions of likely situations, either by preprogramming them or letting the car learn on the job.

The first approach suffers from the problem that it is pretty much impossible to anticipate all possible scenarios – for example, an chasing a duck into the road with a broom, as recorded by a Google car in 2014. The second approach seems more promising. A car might learn as it goes along, for example, that jaywalkers are more likely to be found on city streets than country roads, but that swerving to avoid one on a quiet country road brings less likelihood of hitting something else. Or it might learn that it’s OK to break the speed limit occasionally to make way for an ambulance.

But basic rules still need to be programmed, and whole new ethical issues also arise, says Millar: a programmer will not be able to predict what exactly a car will do in a given situation. We don’t want autonomous vehicles to act unpredictably. Just as it’s important for cars to predict the actions of human road users, so it matters for people to be able to anticipate a car’s behaviour. Hence the question of what an autonomous car will do when it encounters that trolley-problem-like dilemma.

But fixating on such an extreme case probably doesn’t help, says Rojas. “Who has ever experienced that situation? It’s one possible situation in a million days. We first need to solve 99.9 per cent of the more pressing problems” – things like how to avoid pedestrians, stay within a lane, operate safely in bad weather, or push software updates to cars while safeguarding them from hackers. Millar agrees, but says that’s not what the thought experiment is about. “It’s just used to illustrate the point that engineers don’t have the moral authority to make all the decisions in their cars,” he says.

At the moment, companies such as Tesla and Google – which recently announced a withdrawal from building its own cars in favour of supplying software to other manufacturers – work on algorithms behind closed doors, but there are growing calls for transparency and common standards. “We need to move towards some consensus as to what solutions are acceptable,” says . He headed a US Department of Transportation team that, last September, produced the first . It sets out decision-making ethics as one of 15 points the developers of autonomous vehicles should address, and calls on them to be transparent about their work on algorithms that “resolve conflict situations”. It also urges companies to consult to come up with solutions that are “broadly acceptable”. “We’re not really trying to program ethics, but to program ethically,” says Gerdes. Similar calls are being made in the UK. Autonomous vehicles “cannot be expected to make moral decisions around which society provides no agreed guidance”, as Tim Armitage of the industry consortium UK Autodrive put it in a WLG.

Not perfect, just better

Only Apple has publicly commented on the US guidelines so far (and so acknowledged its own driverless car programme for the first time), urging that “.” But, Gerdes stresses, “The intent isn’t for companies to reveal proprietary information. It’s to have enough openness about how cars are programmed to manoeuvre around and respect vulnerable road users.”

That’s a start, but no solution will be perfect, warns and director of its Future of Humanity Institute. “We should accept that some people will be killed by these cars.” That’s where we also need to put things in context, he says. In 2013, people around the world, injuring up to 50 million more. Nine in every 10 accidents result from human factors: a moment’s distraction caused by reading a text message or yelling for the kids to behave, or falling asleep because of monotonous motorways. “Our challenge isn’t to build the perfect system,” says London. “It’s to build a system that is better than the one we have now.”

In Brian Wood’s case, the 21-year-old driver of the oncoming Chevy had been distracted by taking off her sweater. Had her car been fully autonomous, it seems likely Wood would be alive today. In ensuring morals rule on the autonomous road, we may find ourselves meeting driverless vehicles halfway.

This article appeared in print under the headline “Auto correct”