SELF-DRIVING CAR: Researchers at the Massachusetts Institute of Technology asked bystanders if they trust a self-driving car to make life or death decisions on the roads. Image: YouTube
Boston - Imagine you're behind the wheel of your car when your brakes fail.
As you speed toward a crowded crosswalk, you're confronted with an impossible choice: veer right and mow down a large group of elderly people, or veer left into a woman pushing a stroller.
Now imagine you're riding in the back of a self-driving car. How would it decide?
Researchers at the Massachusetts Institute of Technology (MIT) are asking people worldwide how they think a robot car should handle such life-or-death decisions. Their goal is not just for better algorithms and ethical tenets to guide autonomous vehicles, but to understand what it will take for society to accept the vehicles and use them.
'They don't want to get into that car'
Their findings present a dilemma for car makers and governments eager to introduce self-driving vehicles on the promise that they'll be safer than human-controlled cars.
People prefer a self-driving car to act in the greater good, sacrificing its passenger if it can save a crowd of pedestrians. They just don't want to get into that car.
What do you think of driverless cars? Let us know via email or on Facebook and Twitter.
Iyad Rahwan, an associate professor at the MIT Media Lab, said: "There is a real risk that if we don't understand those psychological barriers and address them through regulation and public outreach, we may undermine the entire enterprise.
"People will say they're not comfortable with this. lt would stifle what I think will be a very good thing for humanity."
The Moral Machine
After publishing research last year surveying US residents, Rahwan and colleagues at the University of Toulouse in France and the University of California, Irvine, are now expanding their surveys and looking at how responses vary in different countries.
They are also using a website created by MIT researchers called the Moral Machine , which allows people to play the role of judging who lives or dies. A jaywalking person or several dogs riding in the driverless car? A pregnant woman or a homeless man?
Driverless car crashes: How insurance claims could work in the future
Preliminary, unpublished research based on millions of responses from more than 160 countries shows broad differences between East and West. More prominent in the United States and Europe are judgments that reflect the utilitarian principle of minimizing total harm over all else, Rahwan said.
The thought experiment is familiar to ethicists, who have grappled with such dilemmas since British philosopher Philippa Foot in the 1960s presented a similar question about a runaway trolley . But it's too unrealistic for some focused on how the vehicles act in ordinary situations.
Just 8km from MIT's Media Lab in Cambridge, the first self-driving car to roll out on Massachusetts public roads began testing this month in Boston's Seaport District.
"We approach the problem from a bit more of a practical, engineering perspective," said NuTonomy CEO Karl Iagnemma, whose Cambridge-based company has also piloted self-driving taxis in Singapore.
More: Uber gives riders a preview of driverless future
Iagnemma said the study's moral dilemmas are "vanishingly rare." Designing a safe vehicle, not a "sophisticated ethical creature," is the focus of his engineering team as they tweak the software that guides their electric Renault Zoe past Boston snowbanks.
Iagnemma said: "When a driverless car looks out on the world, it's not able to distinguish the age of a pedestrian or the number of occupants in a car. Even if we wanted to imbue an autonomous vehicle with an ethical engine, we don't have the technical capability today to do so."
Watch: Udacity self-driving car
Focusing too much on the stark "trolley problem" risks marginalising the study of how best to address self-driving ethics, said Noah Goodall, a scientist at the Virginia Transportation Research Council. Engineers already program cars to make moral choices, such as when they slow down and leave space after detecting a bicyclist.
Goodall said: "All these cars do risk management. It just doesn't look like a trolley problem."
Rahwan agrees with self-driving enthusiasts that freeing vehicles from human error could save many lives. But he worries that progress could be stalled without a new social compact that addresses moral trade-offs.
More: Self-driving cars still need humans... to program them
Current traffic laws and human behavioral norms have created "trust that this entire system functions in a way that works in our interests, which is why we're willing to fit into large pieces of metal moving at high speeds," Rahwan said.
"The problem with the new system it has a very distinctive feature: algorithms are making decisions that have very important consequences on human life," he said.