91av

Huddled round a screen showing an animation of a busy street, a group of researchers watch as a child emerges from behind a parked car and is mowed down by a truck. The simulation resets and the same scenario plays out again. And again. Each time, the pedestrian is hit, but in different weather conditions: heavy snow, dense fog or pouring rain.

The animation looks almost like a version of Grand Theft Auto (it is based on the same game engine) but this is no console title. The research team at CARISSMA, a leading research and test centre for vehicle safety in Ingolstadt, Southern Germany, is working on a significant problem for autonomous vehicles. How to make them safer in extreme weather, when their cameras and sensors are impaired by snow, fog or rain.

The CARISSMA team is part of a €9.7 million Europe-wide project called ROADVIEW— a 15-partner research consortium with members from seven countries including the UK, Germany, France and Sweden. All made possible with funding from the EU’s flagship €96 billion research and innovation programme, Horizon Europe.

Extreme weather

The 4-year project brings together some of the best expertise in Europe on autonomous vehicle safety, sensor technology and simulation modelling to achieve results that no single country could achieve on its own. The partners come from universities, research led-companies and car manufacturers, all with the overall goal of solving the extreme weather problem for self-driving vehicles.

“It is very important to bring different partners from different countries together. To bring different thinking together,” says Professor Werner Huber, Head of the CARISSMA Institute of Automated Driving.

Though companies such as Waymo and Tesla have already brought driverless taxis to public roads in the US, their roll-out in Europe has been slower because of a more cautious attitude to safety.

Experience from the US has thrown up situations where the cars have struggled in unusual conditions. In September last year, for example, torrential rain caused acute flooding in Arizona. Driverless Waymo taxis struggled in the conditions with . That led the company to suspend its operations temporarily.

“You don’t want a system that works only in ideal conditions,” says Professor Valentina Donzella, at Queen Mary University of London, whose team of engineers is part of the ROADVIEW consortium, “It’s important to be able to demonstrate safety in adverse weather or complex conditions.”

That’s difficult because of the need to accumulate enough driving experience in extreme scenarios. “You need data, data, data and you can’t get enough [from real world tests] to find out what the car should do in a specific situation,” says Huber. That would be too dangerous on a public road.

The answer is to train the cars’ sensors and control systems in the lab or on the test track using data that simulates extreme weather conditions, something that Huber and Donzella’s teams are working on together. “The UK partners have great experience in this kind of modelling,” says Huber, “It’s not that easy to model bad weather situations into a simulation environment since you need a sensor model which … is really close to reality.”

In simple terms, it means taking real-vehicle motion and reproducing it in a simulation where the data from cameras, LiDAR and radar sensors is degraded to mimic what the car would see in heavy rain, snow or fog. That way, the researchers can take real-world motion and use it in extreme weather tests.

The geographical spread of the ROADVIEW partners confers an advantage, says Donzella. She points out that the nature of fog, snow and rain is not uniform across the continent and that impacts the way autonomous vehicles “see” the road.

“We realised that, depending on the country, we have different types of [fog, rain and snow],” says Donzella, “just discussing with the partners and understanding what we have to model in terms of adverse weather was really interesting.”

In the UK’s maritime climate, for example, snow tends to be wetter than in Finland or Scandinavia. “When it’s dry, it flies more [and] is more affected by wind,” says Donzella, “You have to take that into account [in the modelling].”

Critical scenarios

In Ingolstadt, Huber’s team uses the modelling expertise from Queen Mary to fool real-world cars on the institute’s test track into believing they are driving around in extreme weather conditions. That allows them to repeat critical scenarios like the child crossing the road — but in any weather.

Huber says the collaboration with the UK team has been “very fruitful” with each side bringing different expertise to the table. “UK partners have a special kind of thinking. They are critical and they are innovative and I think German partners are the same, maybe from another perspective,” he says, adding that both sides have long expertise with the automotive industry.

Donzella agrees. “It’s a matter of complementary skills, different expertise,” she says, “It’s been very, very productive.”

The project, which runs over four years, has also provided a structure for deep and long-term collaboration. “It gave us enough time to know each other, understand strengths and how to work together,” says Donzella. Members of her research team have spent time at the CARISSMA test track in Ingolstadt, Germany and researchers from that team have made the reciprocal journey to London.

That has made for a deep and effective partnership, she says. “Obviously we leverage complementary skills, but also complementary facilities, different viewpoints. Some things that can be more challenging here in the UK are maybe less challenging in Germany due to different road infrastructure and so on.”

That’s vital to building an autonomous driving system that will be safe in conditions right across the European continent. “Working together, we can bring this different cross-geographical perspective,” she says.

The ROADVIEW project comes to an end in August, but both sides intend to continue working together. “We have understood that environmental conditions are really challenging … and we need to consider everything together. So the complexity is very high and there are still a lot of things that we need to tackle,” says Donzella.

“I hope this isn’t the end. I hope this is the start of a long cooperation,” says Huber.

Watch to find out more: Inside the partnership helping driverless cars navigate extreme weather 

Find out ����