A software update for the Tesla Model S enables autonomous driving including ‘traffic aware’ cruise control and lane changing. Tesla
Developing the autonomous vehicles that will one day become a common sight on our roadways is no easy task. Autonomy transforms these vehicles into robots, making them much more complicated to build than any car on the streets today.
This presents various challenges the industry has never faced before, and that includes developing the operating systems that control these cars of the not-so-distant future. Some experts say the existing systems that power semi-autonomous vehicles will continue to evolve, and will become more open. The way we maintain these cars will completely change from what we’re used to today, said Dominique Bonte, vice president and general manager B2B for ABI Research, and there’ll be an increased focus on keeping these vehicles safe, secure and reliable.
As we move ever closer to a world where driverless cars are the norm, there’s still a lot for developers to figure out—including devising the best operating systems for these vehicles—the systems that control the car’s many capabilities—while also keeping passengers and the public safe.
“Every component of the car will become software defined and connected, not just the entertainment system, and will need reliability, security and updatability,” Bonte said. “The complexity and scope of maintenance will dramatically increase. Everything needs to become remotely controlled, and software will become the common denominator and key issue to make sure these vehicles can perform safety and security related updates. That’s one of the big changes taking place as we move toward the driverless vehicle of the future.”
An Evolution
Many car manufacturers have already incorporated driverless capabilities into their vehicles, and Bonte said he expects these systems to continue to evolve as cars move toward complete autonomy. There will be a gradual evolution, he said, leveraging the operating systems we already have, rather than several new systems hitting the market.
One of the biggest changes he sees is multiple operating systems co-existing in these vehicles, he said. You might have a system like Linux, for example, coupled with Android or iOS. This will help protect driverless vehicles from cyber-attacks, which is key to keeping the public safe.
“The technology will allow us to run several operating systems next to one another or on top of one another on the same hardware,” Bonte said. “The hardware would drive other parts of the cockpit and the underlying networks of the vehicle. It will be a very diverse and complex environment.”
Tesla dashboard. Tesla
A New Set of Challenges
About 10 years ago, the team at Perrone Robotics developed a very early autonomous operating system. They built it as part of the DARPA Grand Challenge, and CEO Paul Perrone said the obstacles they faced back then were very different from what the industry faces now.
During that time, the team needed to develop a system that could control the car’s brakes, throttle and steering, what Perrone described as low-level problems. Today, the challenge is creating algorithms that work in a variety of environments, such as dealing with rain, sleet or snow, or driving on rural roads versus urban roads. Their system, known as MAX, supports a wide range of algorithms and is adaptable so it can support the algorithms being tested.
“This technology is being used to control automated vehicles and to test automated vehicles at IIHS (Insurance Institute for Highway Safety),” Perrone said. We are actively evolving and constantly integrating these new sensor technologies, both ones we’ve developed and ones from third parties. An operating system has to be able to support and provide services for companies and researchers who have novel algorithms that may be proprietary, so we want to make sure we have a platform that can support those algorithms.”
Tesla control panel with the acceleration set to ‘Ludicrous’. Tesla
High Level of Complexity
Cars today have about 100 different computers on board and run 300 million lines of code, said Josh Hartung, CEO of Harbrick, the company that developed the middleware PolySync, which serves as a layer that allows applications to communicate with each other while giving them the tools to detect threats. If you look at the driverless cars companies like Google and Tesla are developing, these vehicles will need closer to 300 billion lines of code to achieve autonomy. That makes for very complex systems.
The PolySync platform helps OEMs handle the complexity involved with building driverless cars by centralizing their electronics architecture and reducing the number of computers on the car. It makes it possible to build these autonomous vehicles more quickly and more simply, while also providing an open platform.
“The sensors, computers and algorithms it takes to run an autonomous car are an order of magnitude in performance and complexity over what’s ever existed before in automotive,” Hartung said. “They use exotic algorithms that require high performance computers and exotic sensors that cost a lot of money and typically generate really large amounts of data. The challenge is how do we build those systems on a car and keep the cost the same.”
The Benefits of Open Source
Manufacturers will not only have to support driverless car operating systems, but also the application layers on top of the systems, ABI’s Bonte said. Middleware software that acts as a bridge between the operating system and these applications will become more common and more important. And, because developing secure, safe software for driverless vehicles is so complex; it will have to become open source.
“The automotive industry has the tendency to close everything. It’s been effective for them in the past. If you look at the software today, it’s a mix of closed and open interfaces,” Hartung said. “The challenge of developing driverless systems is too big for closed systems. It’s too big of a problem and too important of a problem (for one company) to solve. What we’re working on is a platform that encourages innovation and encourages folks across the industry to focus on what they do really well, and bring that to the community so everyone can benefit from it.”
Automakers tend to keep everything in-house, Hartung said, to give them a competitive edge. If automakers close the ecosystem down and focus on developing their own solutions, it will cause fragmentation of the technology. It will slow down the development process, increasing the time it takes to make driverless vehicles a reality.
The idea of centralization has been buzzing around the auto industry in general for a long time, but it’s a huge cultural shift that also brings safety and security challenges, Hartung said. Autonomous vehicles will likely be what changes this, because the industry just won’t be able to solve the problems driverless cars bring on their own.
Mercedes-Benz is exploring autonomous driving with its F015 luxury concept car. Mercedes-Benz
Platforms like PolySync allow developers to focus on their core value, which is the user experience, while others focus on the details, Hartung said. He expects his system, or one like it, will become a standard for driverless cars.
“We have the opportunity to usher in a new era of applications around human mobility,” Hartung said. “It’s our duty. Think about the airbag. At the end of the day somebody made a lot of money on airbags and three point seat belts, but eventually they crept into every vehicle. They do more than differentiate one car from another; it makes everybody safer. That’s the same with this technology. Maybe 25, 30 years from now you won’t know anyone who knows someone who’s been hurt or killed in a car wreck. It will be a completely different world than what we live in today, and it’s so important to get there.”
Maintaining Driverless Cars
With driverless cars, it becomes even more important to keep them updated to ensure safety, reliability and security, Bonte said. Maintenance will be very important, but it likely won’t involve going to the dealership or a local mechanic when something goes wrong. Instead, these vehicles will receive over-the-air updates.
“I see a transition to almost a fleet based maintenance,” Bonte said. “Nothing can go wrong with driverless technology because lives are at stake. That’s a key change compared to the current generation of vehicles, and a change that will rely on software.”
Over-the-air updates will make car maintenance much easier, Perrone said. When you want your car to talk to a new sensor, you don’t want to have to send it back to the manufacturer to get the sensor installed. You’ll want it done over the air. And while you might have to go somewhere to change out your sensor or camera hardware, you can also get a simple software update as well. This makes the new hardware easy to implement and upgrade in the future. The challenge, of course, is ensuring these over-the-air upgrades are safe and secure.
Today, when designing an operating system, developers try to figure out what could cause the system to fail, Hartung said, and then come up with how to safely respond. The check engine light is a great example of a car’s software detecting a failure and responding. Once failure modes are tested and deemed safe, the car is shipped. There usually isn’t a need to change the software after that point. And every time the software is changed, it has to be re-qualified to ensure it’s safe to drive.
PolySync is middleware, a layer of software that allows applications to communicate with each other while giving them tools to detect threats. Harbrick
Driverless systems will have many more failure modes than today’s cars, Hartung said. The number of different sensing inputs and situations a car can encounter in driverless mode are seemingly endless, and there’s no way to determine them all ahead of time. While most will be accounted for, manufacturers will continue to find failures—making it vital to get software upgrades out to owners right away.
“In a level 4 system (with complete autonomy) that won’t necessarily have a steering wheel (a driver can take over in an emergency), the car will need to be able to respond safely to every possible situation. And in most cases it will, but in some cases it will fail,” Hartung said. “It’s the automaker’s job to catalogue those types of failures as they happen in the field and send out software updates to fix them. Tesla is pioneering this. They send out updates when they want to add features to their system. That’s the future of maintenance of these cars.”
Being able to identify problems and send out updates quickly will involve a whole new way of testing and validating these systems that doesn’t include driving thousands of miles to confirm there isn’t a failure, Hartung said. He envisions a virtual test track that can generate scenarios and faults.
“It might generate 50 million miles of different scenarios to test these systems, and be able to output something that says if the system is safe or not,” Hartung said. “We’re working with customers to bring out these solutions. We have to think about what happens after we ship the car. How do we make sure our system is maintainable?”
The BMW i3 concept car. The firm is one of the many car companies working on autonomous technology. BMW
The Future
This is an exciting time, and there’s a lot of focus from the media on when driverless cars will be available—creating a sense of urgency in the development community, Hartung said. He’s afraid that could lead to manufacturers putting driverless vehicles on the streets before they’re really ready, causing one or more accidents that sets the entire industry back.
These systems must be qualified in such a way that they can guarantee a level of safety, Hartung said. The focus has to be on safety and security, not how soon our cars will be able to drop us off and pick us up from work.
As developers continue to make these systems safe and secure, Perrone said it will be interesting to see how the operating systems evolve. They could become true operating systems or special purpose solutions that are geared toward specific vehicles. Right now, developers are focusing on the algorithms, which require a lot of specialization and testing. Perrone’s software provides a platform that makes the development and testing of those algorithms easier, which is key to moving this technology forward.
“How can you facilitate development and testing of systems so you can make the scalability of these solutions a reality?” Perrone said. “There are also the folks focusing on sensor technology. All these pieces need to come together. There’s really a microcosm of players that need to come together to develop these systems. That’s happening now and will happen more in the future.”
