Computer vision and artificial intelligence (AI) each have their purpose in modern vision systems. When developing an imaging platform, certain challenges may arise when trying to implement AI in an existing computer vision system. Deciding what tasks are best suited for AI can initially be difficult. Therefore, it is beneficial to look to the experience of industry experts who have already taken many steps to provide a combination of AI and computer vision solutions.

Value of AI

Expertise in traditional imaging software has been available for a long time. Using computer vision, engineers have been able to tackle challenges in many industrial applications, including machine vision automation, Intelligent Transportation Systems (ITS), and aerial imaging. However, when images captured are complex and varied, such as new shapes, orientations, and colors, computer vision is known to struggle where AI can flourish.

A common ITS application is vehicle detection. This can be challenging when using a traditional computer vision solution due to the variables that exist in outdoor imaging. In the traditional method, a developer may program specifications so that the software looks for a change in the image to detect a vehicle.  However, if the software triggers a camera when the image changes – to signify a vehicle passing by the camera – several other changes to the image could do the same. For instance, if the weather changes, the software may interpret this as a change that requires the camera to capture an image. Additionally, if something enters the field of view, such as a bird, this may trigger the camera as well.

In a factory setting where the same type of object is continuously passing by the camera, computer vision software can accurately capture images of products, components, or materials for inspection. Conversely, in an outdoor setting, various phenomena can introduce inconsistency for the scene and cause complications when programming a solution. AI can provide value in such a situation.

Training AI software, which is an algorithm that identifies a specific subject, is quite different from programing computer vision software. The process of training an AI software is less rigid. It is similar to introducing a subject to a student and having them review the material many times until they understand. However, AI needs to be focused with a particular subject in mind.

Instead of training AI software to be able to recognize everything it sees with the camera and to identify vehicles, the training is focused on only vehicles. By recognizing what a car, truck, or motorcycle looks like, an AI program will know to look out for possible targets within an image. With this baseline, AI software can ignore all other issues that may arise in the real-world environment of traffic imaging.

Value of Computer Vision

While AI can provide new capability to a vision system, it is not the only option. Traditional computer vision software can offer efficient solutions to various image challenges. Where AI can be beneficial for complex image analysis, it also comes with a drawback, the amount of strain it puts on computer resources. On the other hand, computer vision software offers simple programs that are easy to run on some of the smallest embedded systems.

Returning to traffic applications, one way they can greatly benefit from computer vision software is with optical character recognition (OCR). Instead of using AI that might require more computer resources, a computer vision OCR software simply cross-checks the image for basic shapes to match one of a pre-set list of characters for license plates. Characters from each license plate in Figure 3 are passed through an OCR system and printed out for data collection of each vehicle that is captured by the camera system. This can be advantageous for effectively registering each license plate by scanning each character and saving that data for toll enforcement.  

How to Use Them Together

A vision system that uses AI plus traditional computer vision can accomplish a much more complicated image analysis.

For ITS, both OCR and vehicle detection are crucial for toll enforcement. But to ensure the computer vision software is able to reliably pass through a consistent image sample, the image system would also need to isolate each vehicle in an image. In any one image the camera may capture several vehicles with each license plate needing to be separately isolated and detected before being inspected by the computer vision software (See Figure 4).

Since every vehicle can have a variety of license plate positions and they can also take on a variety of colors, it can be advantageous to have an AI that is used to these edge cases. A license plate from a particular region can come in multiple colors because as new license plates are issued, they can have new styles that replace old formats. In Figure 5, different license plates are shown from various regions and even some from the same region (Ontario) with different designs that could confuse a vision system that is not well trained to accommodate these options.

Once a license plate is detected, the rest of the image can be cropped out so that only a rectangular license plate remains from which the OCR can be used to extract the characters. The three license plates in Figure 6 show various forms of dirt or mud that can obscure a license plate. Even license plate covers can reduce the light reflecting from the characters and can provide an additional challenge.

The Real World Example

Implementing AI into an existing vision system can be challenging. There is an ITS imaging system for OCR which was initially designed to be a computer vision system with a goal of achieving a high accuracy and was later expanded to incorporate the benefits of AI.

The reality of this vehicle detection system was that there were too many variables for computer vision alone. AI was introduced to capture the images, and computer vision ended up providing efficient analysis for OCR. The resulting system involved a lot of work in developing and testing different options.  

Many people may try and set systems up the traditional way and just use computer vision. However, AI allows systems to easily detect targets in complex images, while computer vision can quickly work through repeatable image analysis. Imaging experts have demonstrated that both AI and computer vision are important tools in modern imaging systems and work with system integrators to provide elements of vision systems that combine the best of both worlds.