By Jeff Sanford
Toronto, Ontario — May 16, 2017 — Organizers promised the latest Guild 21 conference call would be provocative. Listeners were not disappointed. The special guest presenter, Hod Lipson, answered the one question on everyone’s mind: “When will autonomous vehicles (AVs) really arrive?”
His short answer: “Ten years from now.” At that point the AV era gets underway in earnest. The 30 years that follow will likely see the auto industry evolve into something entirely new.
Interestingly enough, Lipson predicts the demand for vehicle maintenance services will grow. The number of kilometres driven will expand. The number of people accessing AV services will rise as the total pool of “drivers” expands to include more seniors and the disabled.
There is no question that Lipson is qualified on the topic. A professor of engineering and data science at New York’s Columbia University, he is the co-author of the book Driverless: Intelligent Cars and the Road Ahead. He directs Columbia’s Creative Machine Labs. He has co-founded four companies. His TED talk on self-replicating and self-aware machines is one of the most viewed presentations on AI and robotics. According to the host of the Guild 21 call, “He has a robot at home that is learning how to paint pictures based on its experience.” In short, this is someone who knows robotics.
According to Lipson, events are suddenly moving rapidly ahead in the sector. Decades of patient and quiet work is beginning to pay off. “It has been surprising people in the industry and academia. It’s moving so quickly. It’s taking us by surprise,” says Lipson. “We’ve had lots of fits and starts over the decades, but now it’s happening.”
The journey to this era of AVs is the culmination of a long arc in auto industry history. Lipson notes a historical fact not always appreciated: AVs were being discussed from the very start of the auto age.
“Soon after cars were introduced and before they were widely adopted people were already dreaming about driverless cars,” says Lipson. “They were dangerous, there were accidents.”
The 1919 World’s Fair in Queens, New York featured a GM Pavilion with a “future city” that was the most visited pavilion at the fair that year, according to Lipson. Visitors enjoyed a view of what a future city might look like.
“One particularly interesting thing was that it had driverless cars. People were already thinking in the ’30s that cars would become driverless and that the technology would soon arrive to do that. Interestingly, people then thought that in the ’50s these driverless cars would be driving around,” says Lipson. “That didn’t happen.”
World War II came and went. By the 1950s people “were again dreaming of AVs.” Technologies developed during the war found new uses in the domestic economy. RCA built a vehicle without a steering wheel that tracked a buried wire. A UK research lab did something similar. The tech worked in that form. But there were other priorities that consumed the auto industry. Safety became a bigger issue after the Ford Pinto debacle. Fuel efficiency became a focus.
The industry, by and large, seemed to forget about AVs. But research continued over the decades. The robotics industry diligently chipped away at outstanding problems. Important hurdles were cleared. The technological pieces are now in place.
As Lipson explains, automated low level vehicle control—move the car forward or backward, change speed (drive-by-wire)—was accomplished decades ago. High level control of a vehicle—how to get from New York to, say, Los Angeles, using GPS to navigate—is also in place.
“Everyone uses these technologies today. We perfected that years ago,” says Lipson. What was missing was the “mid-level control,” the minute-by-minute control scale. “That had not been solved. Merging into traffic. What’s a pothole? What’s an oil spill? Avoidance. How to negotiate a bustling intersection? These things had been virtually impossible for any AI to master. These mid-level controls were the biggest stumbling blocks, the last piece. And this is the part that’s been solved in the last couple of years,” says Lipson.
There are two ways of building AI systems. The first is for programmers to write a list of rules for the program to follow. This was the approach used for the first 40 years of AV development. It worked in the lab … but nowhere else.
“It was quite easy to get a robot a round a clean lab with neat obstacles. You could write papers and students could get Phds, but when they took these rule-based robots to the real world, they did not work. That turned out to be the challenge … You can write rules to do all sorts of things. But it’s tough to write rules for what’s drivable and what’s not drivable,” says Lipson.
The other way to approach AI is to use a method called machine learning, which sees computers develop what can be considered “computer experience” based on data taken in. Says Lipson, “This other approach has been, basically, ‘Don’t program the computer…show it. Give it an example.’ How do you learn a new language? People learn by example. We’ve been doing the same thing with computers through machine learning. Now people are building these small widgets that can drive around in the lab. They are growing quickly because of the things they keep learning.”
This shift in approach, from rules-based programming to machine learning, has been the final piece in the puzzle that has allowed modern AVs to emerge over the last couple of years. “What makes a driverless car is the control system. These new systems are learning from examples. What they didn’t have when AVs were researched in the ’50s was enough data or memory to do machine learning. Now we have enough memory and the machines.”
According to Lipson, algorithms compose the basic formulas that will pilot an AV. But it is the data feeds those algorithms (including the experiences of the car in the past, or other cars) that is allowing AVs to drive in the real world. “The data is the fuel. The algorithm is the engine,” says Lipson. “With a good engine and enough fuel you can take off. And now we have cheaper and faster computers and we have tons of data. It is the data that is fueling this AI revolution. With an accumulation of data, and faster computers, computers can finally understand what they are seeing. That has caught the AI community by surprise. AI can look at an image and understand a dog, chair and person. That was not possible just five years ago. It’s possible today. We drive different with a toddler on the side of the road or a fire hydrant. Five years ago cars couldn’t understand the difference between a child and hydrant. Five years ago they couldn’t do that. Now they can.”
According to Lipson, around 2014, the number of cameras on cars began exploding. This has also been key.
“Cameras generate data for AI systems to learn on. AI systems keep getting better and better the more data they get. Because of these sensors the imminent revolution about to happen is driverless cars. The last piece of the puzzle was the data to learn on,” he says.
The start of this new era might be traced to somewhere between 2004 and 2006. In ’04 DARPA (the US research foundation that created the Internet) started a competition. The organization was going to give $1 million to anyone who could drive a car across the desert with no one inside.
“No one knew where the track was going to be. They gave out the route day before,” says Lipson. “Lots of people applied for that prize. But no car went more than seven miles. It was a big embarrassment. It was clear you can’t ‘tell’ the car what’s drivable. Cornell’s car halted in front of an overpass. The AI did not know it could drive underneath. The car just stopped there, confused. Turns it is difficult to program a list of rules. You can’t program the myriad ways that reality can [occur],” says Lipson.
The next round of the race was held in ’06. “They doubled the jackpot to $2 million. Teams were more cautious. Hundreds of teams submitted applications. Schools, big companies, small companies, hackers, students writing code … all kinds of people were involved,” according to Lipson. That year, in 2006, five cars finished and the Stanford team won. “In the end they started to rely on machine learning. They taught it. They didn’t program rules. They drove it around and showed it how to drive,” says Lipson. The new AV era was upon us.
According to Lipson, Google has an early lead in the AV race. “All this tech is open source. It’s free. Anyone can build this. Anyone can build a car … but it is difficult to accumulate the data to train the AI system. Google’s driverless cars have accumulated two million miles of driverless data. They’ve been driving around collecting that fuel that makes these things good. They’re in the lead. It’s not a better car mechanically, but they have these algorithms that can learn. That’s the big differentiator,” says Lipson.
Taking a poll of the audience, Lipson found that, interestingly, the crowd of collision repair industry executives on the call tended to think AVs would arrive sooner than most other audiences. “It’s interesting, the number of people who say that AVs will be here in less than 10 years is usually less than that … If you ask me, I think it’s going to be about 10 years,” said Lipson. “And then it will be 30 years after that before most cars are autonomous.”
As for the future of the auto industry, “When it comes to the inside of the car, people are beginning to re-imagine that. What are people going to be doing inside cars? People are getting excited about the rethinking of the car. We’re going to see all kinds of new designs,” he says.
Lipson also notes that the arrival of AVs will see vehicles driven far more hours a day than cars currently are. This will generate new levels of demand for maintenance work. There will be fewer accidents (even though there will always be body work to do). But new groups of vehicle users—the elderly, the disabled—will also drive demand and create large demand for maintenance. And so the total amount of repair to be done will continue to grow.
“If you are in car maintenance it’s going to be a more interesting and more complex world. A lot of people who can’t use a car today will be able to use cars. A larger section of the population will use cars and will spend more time driving. Anyone in auto industry should be overjoyed. There is going to be more mechanical work. All the jobs around maintaining cars will increase because there will be more cars being used,” says Lipson. “It seems likely there will be fewer collision repairs, but there will be many more mechanical repairs.”
There will also be a huge new need for software and calibration work in the years after 2027 when the AV era truly arrives.
“This will change the industry in profound ways. And there is going to be change. But it’s going to be amazing. Once you can take the driver out … you can have a much broader range of vehicles,” says Lipson. “Driverless can bring repair parts on the fly. There will be more miles driven per person. It’s an exciting time. People will re-imagine the vehicle.”
Another interesting idea from Lipson’s address: Because cars will learn how to drive, they will never be perfectly safe. They’ll just be safer than a human driver. Says Lipson, “In the future, a car will have a third rating after horsepower and mileage. ‘How much safer is this than a human?’ People will pay more for a certain safety rating, for a car that is rated 80 percent safer than a human. That will be a number that people will want to know.”
