Intro to Self-Driving Cars

Written by TKS New York student, Sabeeh Hassany (email:

Imagine this: You’re sitting in a cafe and just finished that final report for Q3 sales. You’re getting ready to leave and make your way to your next meeting downtown. Setting out the cafe, you head to the street and call a ride, as you normally would. But this time the car is empty. No, not getting lucky with UberPool empty, but no driver empty.

But you knew that before calling the ride. It’s the future and autonomous vehicles are officially out on the road being used by millions every day for safe and fast commutes.

Surprisingly, this future is here. With advancements in the field of autonomous vehicles over the past decades, many companies have risen in the space including Waymo, Cruise, and even some familiar names such as Uber.

However — like with any new technology — people are very confused, and frankly very uninformed.

The problem is that self-driving is such an ambiguous term. It can easily be generalized into driver assisting systems or fully autonomous cars: two opposite ends of a spectrum. But it’s far more expansive than that.

But first, how does it even work?

There are five core components that enable autonomous driving. These five components come together and collectively allow a car to safely maneuver streets without any risk to people or objects inside the car and outside.

The first is computer vision.

The same way humans have normal vision using our eyes, autonomous vehicles use cameras to achieve a similar feat. By using cameras and deep learning neural networks, a self-driving car can find lanes, street signs, speed limits, and more.

Example of computer vision in a New York City street *whoop whoop*

The second component is sensor fusion.

As the name implies, this is utilizing other sensors, such as radar and lasers, to build a more accurate and safe virtual environment. This technique makes up for the shortcomings of cameras such as gauging distance, depth, and velocity. This also gives the car critical data during adverse weather conditions, when just the camera would not have given enough information.

Camera + LIDAR + Radar = Self-Driving🚗

The third component is localization.

This is essentially how a self-driving car figures out where it is in time and space. While we can easily resort to GPS for getting around, self-driving cars cannot. This is because GPS signals are only accurate in localizing you within about 1–2 meters. That might be okay for humans to get around but for a car, even half a meter can lead to riding on the curb or hitting another car.

Due to this, self-driving cars utilize sophisticated mathematical algorithms to localize themselves using reference points in real life to localize itself. It’s almost like a constantly moving and changing spider web. With the car being the exact center of the web that is focalized to all ends of the web.

The fourth component is path planning.

Once the car takes in all the data it received from the previous components, it is ready to plan its path. Path planning is when the car calculates what other vehicles around it will do, what the driving rules and limits are, and similar metrics to determine an optimal path to travel on.

The onboard computer in self-driving cars does this stage multiple times per second to ensure that any sudden and dangerous event can be accounted for. Whether it be a car disobeying a stop sign or a sudden lane switch, path planning makes sure that the car and passengers are safe and are traveling on the optimal path.

And finally, the fifth component is control.

This is probably the most simple, yet most important, component. This is where the computer calculates how to turn the wheel and hit the gas in order to safely execute the path. Surprisingly (or not a surprise if you’re a computer scientist), computers are much better at this than humans and are constantly improving and getting better.

All these components come together perfectly, like a Power Rangers Megazord, to achieve fully autonomous control and lead to what we call a self-driving car. In reality, it’s not really a SELF-driving car but rather, a COMPUTER-driven car. Although, self-driving does have a better ring to it…

Nonetheless, self-driving is quite a large spectrum of cars. So to save you from being roasted when you call cruising mode self-driving, lets break it down.

5 Levels of Self-Driving

All Work, no Fun: Level 0

  • Driver does all the driving (steering, throttle, brake, etc.)
  • What’s been going on for nearly a century

Single Function Driver Assistance: Level 1

  • An advanced driver assistance system (ADAS) can assist the driver in only one function like steering or acceleration
  • Mostly features safety-oriented features such as automatic lane keeping
  • Driver must remain in full control of the car

Double Function Driver Assistance: Level 2

  • An ADAS can assist the driver in two function like steering and acceleration
  • Can include cruise control and lane centering
  • Driver can have both hands and feet off the controls
  • Driver must be ready to take full control

Conditional Autonomy: Level 3

  • An ADAS can perform all driving task under certain conditions
  • Driver can transfer all safety-critical functions to the vehicle only under certain traffic or environmental conditions and geofenced areas (i.e. a highway)
  • Driver must intervene when asked
  • Does NOT monitor the environment fully
  • Utilizes primarily cameras and computer vision
  • Tesla’s Autopilot

Yes, that dot is the Tesla

Full Autonomy: Level 4

  • An ADAS can perform all driving tasks AND monitor the environment
  • Can only handle certain speeds and terrains
  • Driver does not need to be in control at all
  • Utilizes sensors, cameras, and computer vision
  • Waymo

Waymo’s self-driving taxi service car

Ultimate Autonomy: Level 5

  • Driver is irrelevant and non existent
  • Everyone in vehicle is a passenger
  • Steering wheel and controls don’t exist

If that was too much, here’s a quick visual to summarize:

Okay, But Why Should You Care?

Now that you know so much more about autonomous vehicles you might be wondering why? Many people enjoy driving their cars plus most people don’t want a mindless computer behind the wheel. I know I wouldn’t.

But the thing is, most of the times that computer isn’t actually mindless and is in fact much smarter and better of a driver than us. Over 90% of vehicle crashes are due to human error and many times these crashes can be fatal. In 2017 alone, 37,133 people were killed in motor vehicle crashes in America.

Thats almost the amount of people that died from gun violence. Imagine that. Motor vehicle crashes are nearly as dangerous as guns in America, yet we don’t give them half as much attention. Crazy.

Not to mention, there’s a whole ‘lotta money lost for every crash.

A National Highway Traffic Safety Administration study showed that motor vehicle crashes in 2010 cost $242 billion in economic activity, $57.6 billion in lost workplace productivity, and $594 billion due to loss of life and decreased quality of life due to injuries.

The National Highway Traffic Safety Administration

Autonomous vehicles can easily decrease that number with their increased safety and mobility and save billions of dollars and, more importantly, thousands of precious lives. That’s why it’s so important that we are educated on emerging technologies, such as autonomous vehicles, that can decreases human suffering and make life a hell of a lot easier.

Plus, I would love to be able to drive to school and also do my homework, at the SAME TIME.

Key Takeaways

  1. Autonomous Vehicles, or interchangeably self-driving cars, are in many cases far safer and efficient than a human driver. 😇
  2. There are also many different types of autonomous vehicles (one size does not fit all in this case). 📏
  3. Autonomous Vehicles can save billions of dollars and thousands of lives! 🙌