Taking their hands off the steering wheel
2 CMU researchers pioneered self-driving tech
At the end of July 1995, a pair of Carnegie Mellon University researchers embarked on a nearly 3,000-mile road trip across the country — from Pittsburgh to San Diego — in an unsuspecting black minivan.
Neither was driving.
The van, a 1990 Pontiac Trans Sport dubbed “Navlab 5,” was loaded up with the latest self-driving technology developed at CMU. Its pilot was called the Rapidly Adapting Lateral Position Handler, or, RALPH for short, and it had the wheel as Dean Pomerleau and Todd Jochem kept a careful eye.
Their journey was called the “No Hands Across America” tour, a play on the 1986 “Hands Across America” fundraiser, and it was the longest trip ever made by a selfdriving vehicle at the time — 2,849 miles over seven days from July 23 to 30 in 1995.
Thursday was the 25th anniversary of Navlab 5’s arrival in San Diego.
The “No Hands” trip showed self-driving technology was not only possible in everyday commercial vehicles, but it could also safely handle long distances.
On top of that, it cemented CMU as a leader in self-driving robotic technology and earned the school a grant from the U.S. Department of Transportation for the Automated Highway System project. No other research group had come close to such a long journey in an autonomous vehicle.
“We thought, ‘What would make a splash and get people’s attention?’ We came up with this ‘No Hands Across America’ trip and tried to make it as mediafriendly as possible,” Mr. Pomerleau said Friday.
At the time, Mr. Pomerleau had just earned his doctorate in computer science and robotics while Mr. Jochem was still a doctorate student.
The two made stops at landmarks like the Gateway Arch, the Hoover Dam and the Indianapolis Motor Speedway. They gave out Tshirts and provided demonstrations. The van even got a look from Jay Leno, who was the host of NBC’s “The Tonight Show” at the time.
“We were hoping to get on [the show] but didn’t make it up on stage,” Mr. Pomerleau recalled.
Small road trip, giant leap
Once Mr. Pomerleau and Mr. Jochem finished the trip, the capability of selfdriving technology increased exponentially.
“From 1995 to the end of 1997, in those 36 months the technology went from sortof-driving OK in very simple situations to being able to drive — steering, throttle, brakes — on the highway so well that a person driving next to you would never know a computer was driving,” Mr. Jochem said.
The effects of the trip are felt today in Pittsburgh, where companies such as Argo AI and Uber have been testing self-driving cars for the past several years with the hopes of one day developing technology ready to ferry travelers completely handsfree.
On Thursday, the U.S. Department of Transportation announced the University of Pittsburgh would receive a $1 million grant to study the accessibility and implications of autonomous vehicles for people with disabilities.
The roots of the “No Hands” trip stretch to the late 1970s, when Carnegie Mellon’s Robotics Institute was still a fledgling program. Chuck Thorpe, who arrived as a graduate student there in 1979, recalled watching a primitive self-driving robot designed by Hans Moravec, which worked by analyzing several pictures of a path before deciding how to move.
“It would take the robot 15 minutes to process pictures and move one meter,” said Mr. Thorpe, who is now the dean of arts and sciences at Clarkson University in Potsdam, N.Y. “That got us into mobile robots, mostly indoors.”
In 1984, Carnegie Mellon applied for and won a grant from the United States Defense Advanced Research Projects Agency.
DARPA had just launched its Strategic Computing Initiative, a $600 million project designed to modernize the world of computer and robot technology for both military and commercial usage.
“The Army wanted a robot scout,” Mr. Thorpe said. “The most dangerous job on the battlefield is the scout because, by definition, they don’t know what’s out there.”
It was a big order. The machine had to be able to follow roads, avoid obstacles, and have multiple sensors while carrying all of its computers on board.
“This was pushing all the frontiers at once. It was really a pretty revolutionary and forward-looking set of tasks,” Mr. Thorpe said.
That got the team at CMU going.
Infancy of self-driving
There are many ways to get a self-driving vehicle to follow a road. CMU researchers first worked with video input systems, feeding color and monochrome video into a computer, which would decide where a road or path was.
The first Navlab vehicle was designed in 1986, using SCARF, or, “supervision classification applied to road following,” and YARF, or, “yet another road follower,” systems to analyze and find different varieties of roads.
But those systems could be confused by difficult lighting conditions outside and took awhile for the computer to process. When he arrived at CMU in 1987 as a graduate student, Mr. Pomerleau designed the ALVINN system, short for “autonomous land vehicle in a neural net.”
Fans of the “Terminator” franchise are familiar with the concept of neural net processors. In essence, they are computers able to process and learn from information they are “fed,” and they opened up new opportunities for the team at CMU.
“My hypothesis was: These neural networks were supposed to be able to learn from example. Why don’t we try to drive with one?” Mr. Pomerleau said.
“All it was saying was, ‘If the image looks like this, turn the wheel like that,’ ” Mr. Thorpe said. “From there, they could work on integrating that with map following and obstacle avoidance.”
ALVINN was much faster than SCARF and YARF but still could be confused with transitions from different kinds of roads and lose self -driving guidance. Finally, Mr. Pomerleau and the team designed RALPH, which attempted to combine the best attributes of earlier systems.
“We distilled what the neural network was picking up in the ALVINN system into a more efficient and flexible system that could rapidly adapt to a new road system,” Mr. Pomerleau said.
RALPH used a camera system trained to look for the middle of the lane on a road and ignore things like oil slicks, ruts and tire marks. The camera system then could connect to a steering system and drive the car — a solution much easier and more effective than its predecessors.
“That’s one of the advantages of RALPH; it was more efficient than the generic neural net,” Mr. Pomerleau said. “It was able to run on a really early laptop, which could run off the cigarette lighter [in a car] as opposed to a 5,000-watt generator.”
This meant that any vehicle could be configured to use self-driving technology.
“It went from being able to operate on very well-lined streets to being able to steer like a human would on a wide variety of roadways,” Mr. Jochem said.
Four months later, RALPH was leading Navlab 5 and its occupants through the heartland of America, directly controlling the Pontiac minivan for 98% of the journey.
The van was likely scrapped long ago, but RALPH lives on in modern lane departure systems in commercial vehicles such as those used by shipping giant FedEx. Mr. Jochem and Mr. Pomerleau formed AssistWare to refine the system for commercial usage.
“That was the first commercial spinoff of self-driving technology in Pittsburgh,” Mr. Pomerleau said. AssistWare was sold to the Takata Corporation in 2006.
The two are optimistic about the possibility of automated vehicles helping people with disabilities, which Pitt will be studying.
“There’s more to be done in urban situations, but when this is fully capable, persons with disabilities will be able to have their whole world expanded,” Mr. Jochem said. “If you can’t use your legs, these cars have — or soon will have — superhuman capabilities to make sure you can drive safely.”
With that said, Mr. Jochem believes that fully self-driving cars are still down the road.
“I don’t know exactly where I stand on this. I’m not saying autonomous cars have to be better than humans in all situations, but in a deployment situation they have to be at least as good as humans, and they’re not there yet,” he said.
“What we didn’t envision at the time was this ridesharing mobility angle,” Mr. Pomerleau added. “That technology is still pretty expensive, tens of thousands of dollars per vehicle. It’s hard to justify in a single-ownership vehicle.”
Mr. Jochem now cruises around in a Tesla, which will be able to drive itself in the future once Tesla’s “Autopilot” program is approved. The software is very similar to what he worked on at CMU.
While it might not be able to drive itself around as well as the Navlab 5 could, his Tesla has a few advantages over the old Pontiac Trans Sport.
“It’s certainly a lot more comfortable,” he said.