When we drive, we use our eyes to decide where to go. The lines on the road that show us where the lanes act as our constant reference for where to steer the vehicle. Naturally, one of the first things we would like to do in developing a self-driving car is to automatically detect lane lines using an algorithm.
In this project, I will detect lane lines in images using Python and OpenCV. OpenCV means “Open-Source Computer Vision”, which is a package that has many useful tools for analyzing images.
My code for this project is publicly available and can be found here:
- Make a pipeline that finds lane lines on the road
In software engineering, a pipeline consists of a chain of processing elements, arranged so that the output of each element is the input of the next; the name is by analogy to a physical pipeline.
So my pipeline consists of the following six major steps:
- Gaussian Blur
- Canny Edge Detection
- Region of Interest
- Hough Transform
- Draw Lines
Let’s use the image below as our original image being fed into the pipeline:
Converting images to grayscale returns an image with only one color channel. Not only does this reduce the amount of data, it also lowers the complexity we are dealing with. Grayscaling the image benefits our pipeline further down in Canny Edge Detection.
2. Gaussian Blur
Before edge detection, we want to smooth out the image. By applying Gaussian Blur we will cut down on visual noise and ensure that only the sharpest edges get through.
3. Canny Edge Detection
Canny edge detection is an algorithm that measures the change in connected pixel values, the gradient. We use it to turn our images into pure black and white where the white color represents the largest gradients, meaning where the most drastic changes in connected pixel values occurred. These represent our edges in the original image.
4. Region of Interest
An image mask is applied to the region we are interested in, that being the lane lines on the bottom half of the image. To create an image mask we had to hard-code the coordinates that we are interested in to create a trapezium shape. Giving the following results as shown in the image below.
One of the downfalls of hard-coding the values are if the car is going up a hill or coming out of a valley, the horizon of the road will change vertically. A function can be created to dynamically determine the region of interest on a per-image basis that accounts for these varying conditions. However, for this project, those various conditions were not included and hard-coded values worked fine.
5. Hough Transform
The simplest way of explaining Hough Transform is that it will scan the image and detect where pixels form lines. If we set Hough Transform parameters correctly, this will return our lane lines.
What Hough Transform will do is create multiple Hough Lines. An average lines function will take a line from Hough Transform and reduce it to two lines, that would be our lane lines. This is mostly determined by the negative and positive slopes of the lines and ignoring when slopes become too horizontal (slope = 0) or vertical (slope = infinity). From the separated line information, we average the points together and extrapolate out to the top of our region of interest and the bottom of the screen. This gives us two solid lane lines.
6. Draw Lines
We finally draw lines from Hough Transform onto the original image. This pipeline is drawing on a single image, but it could also work for a video. A video is a series of images stacked together and each image would have its own lane line drawn again.
Here is some video footage with my drawn lane lines:
Potential shortcoming with the current pipeline
The model will struggle to detect the lane lines in a poorly lit condition. This is because the parameters are fine-tuned to detect lane lines in those specific conditions (test video). Another shortcoming would be that it can only detect straight lines, therefore on a curved road, it may not detect the lane lines or it may misclassify.