What’s So Hard About Making a Self Driving Car?

If only I can live long enough so that a car can drive me to where I need to be—while I am in the back seat reading a good book. We, as a society, may have gotten over our love affair with the auto—a few auto aficionados and gear heads excepted. We don’t yearn to drive. For the youth, the first car may no longer be a rite of passage. Driving seems to be interfering with what we really want to be doing: texting, watching YouTube, tweeting.

Perhaps, recognizing that the public’s infatuation with driving is over, every major car company is getting into autonomous vehicles. As are non-auto companies. Both Apple and Google have declared their intentions in this arena. Google, on its own, may very well be in the lead for miles driven by an autonomous vehicle.

What It Takes

If you are wondering why billion-dollar auto companies are having such a hard time realizing a fully autonomous road vehicle, and actually making a safe self-driving car, consider the process human beings have to go through before they can pass a driving test.

1. Process visual information—this takes most of our early infancy
2. Feedback and reaction
3. Hand-eye motor skills

Let’s not forget that when driving, we are also negotiating at superhuman and lethal speeds with what is, in effect, a volatile mass of metal. Let’s assume theses kills can be programmed in. Regardless, the steps above, which we take as a given, were really learned over years, in a manner that is not entirely understood and has never been adequately duplicated and reproduced in either machine or computer.

Then, it’s really no wonder why the quest for robots that can stack crates, negotiate uneven terrain, including steps, and interact unflinchingly with humans—all things we take for granted—is full of spectacular fails. And those are easy tasks. Now we want a machine to be able to go down a narrow two lane road, with another car approaching, a closing speed of 120 mph, and a baby in the back… Why do we think that can ever be possible?

Image Processing

As it turns out, the secret rests in being able to make sense of images. Seeing the picture ahead of us on a road is a learned skill, more complex that you might have realized, simple only until you try to program a computer to do it. The computer will have to rely on a sensor, a digital camera, to generate an image. The image will have to be analyzed  and pattern matched so that the computer can recognize things. What is a car? What is a telephone pole? Where is the dividing line? You can do that in microseconds now but it took years for your eye/brain combo to develop that ability for what is essentially reflected electromagnetic radiation. Oh, the computer will be required to do all that every ninetieth of a second. Then the next image is upon it. Not only does this next image have to be processed the same way, discerning all significant objects, but now the computer has to process the changes. The changes have to be translated into speed . . . and threats. Is the oncoming vehicle simply passing by, or on a collision course?

Threats are not always external. Inside the car, photographs may reveal that the driver is nodding off. Shall the computer slow down the car and pull off to the right, or blare the radio to wake up the driver?

The understanding of pixelated  data, making it recognizable as meaningful objects, such as cars, lampposts, lane dividers, trees, etc., all at close to a hundred times a second for frames that are megabytes in size . . . . Don’t try that on your home computer.

What kind of horsepower does it take? HP’s VR workstations, which can output high resolution images in a surround environment, start at $4,300. That almost doubles conventional workstation prices. And it may be just a start. Processing, analyzing and executing in 3D environment will smoke your computer. Literally.

Most, if not all, of the processing happens in the GPU, or the graphical processing unit, say several hardware vendors. Sure enough, a workstation cranking away at a compute-intensive task can heat up a bank of graphics cards to the point at which getting rid of excessive heat is a limiting factor. One scientific application (virus modeling) at GTC  generated so much heat, the heat sinks around the cards became too hot to touch even with the fans going full blast.