Today’s vision systems have many advantages if you have the right lens for the job. Applications such as autonomous mobile robots (AMRs), bin picking, stereo vision, and agricultural monitoring all require wide angle lenses. Until recently, only fisheye lenses were available for imaging ultra-wide fields of view. But fisheye lenses have high levels of barrel distortion, affecting resolution at image edges, ultimately limiting the application uses for these lenses.

Only rectilinear lenses provide ultra-wide fields of view and correct the barrel distortion inherent in traditional wide angle lenses without losing resolution. This report presents the challenges of using traditional wide angle lenses and discusses technological innovations to address these issues to advance the practical use of wide angle optics in machine vision applications.

Wide Angle Lens Design

While representing the three-dimensional world on a two-dimensional plane, there are two common approaches to designing wide angle lenses which yield different optical effects:

1. Equal angular slices: each pixel receives an equal angle leading to barrel distortion in the image – this is a ‘fisheye’ style lens.

2. Equal planar distances: each pixel images an equal distance in a plane – this is a rectilinear lens.

Rectilinear lenses keep straight lines in the real world straight on the image sensor. One effect of this is an effect called lean-over (or keystoning) when two parallel lines seem to converge at the horizon. Since these two lines are imaged as straight lines on the sensor, they will appear to converge in a keystone shape as they move farther from the camera. This looks like the object is leaning away from the viewer. One exception, and where rectilinear lenses excel, is when the lines are in a different plane perpendicular to the optical axis such as imaging an LCD screen for defects. In this case, the straight lines of the LCD object will be imaged as straight lines on the image sensor. Each pixel of the image sensor will cover the same area of the LCD object and the image will appear rectilinear. If the same object were imaged by an equiangular fisheye lens, the edges of the LCD would be curved and compressed. An area of the LCD object at the image edge would cover far fewer image sensor pixels than the same size area at the center of the LCD, unlike the rectilinear case.

Another noticeable detail of this lean-over effect is caused by objects at the edges of the wide field of view of a rectilinear lens. Because of the seeming convergence of parallel lines these objects at the image edge show 3D stretching, in which objects at the image edge seem to be stretched because they are being “flattened” onto a plane along the tangent angle from the lens.

The wider the field of view (for rectilinear lenses), the more noticeable the effect. In the image below, captured with a rectilinear lens, both cars are the same width but since Car B is seen much more obliquely it appears to be stretched. The cars are the same width because they are the same plane perpendicular to the camera and the rectilinear lens images these equal planar measurements to equal distances on the sensor. The length of the car is stretched due to flattening onto the sensor plane.

The added benefit of this is increased edge resolution compared to equiangular lenses. This increase in resolution allows for more accurate object detection, identification, and positioning, vital to situational awareness and navigation applications that require accurate and simultaneous localization and mapping.

Optical Distortion

Typical wide angle lenses with fields of view greater than 80 degrees, including fisheye style lenses have the well-known, curved barrel distortion look. This distortion results because the lens images with equal angular slices on the sensor. This causes the image to be compressed and look curved and the resolution is reduced as the object moves farther away from the center of the image (see Figure 2). Objects at the edges of the image are compressed and the detail information is lost. The information was lost travelling through the lens and no software manipulation will be able to recapture the lost information.

Barrel distortion can be corrected with software (creating a rectilinear lens image), but at a cost in time or processing power. These costs become particularly important in real-time applications such as autonomous mobile robots (AMRs) and Un-crewed vehicle (UV) navigation.

On the other hand, a rectilinear lens offers the benefit of having no latency – no correction is required - and so no delay is introduced in the system to achieve the ultra wide angle images. In a rectilinear lens, barrel distortion is corrected optically in the design of the lens without the need for software intervention.

Technological Innovation:

To address the challenges of using ultra wide angle optics in industrial applications a rectilinear lens can be used. One such rectilinear lens technology has been developed and commercialized; this lens removes barrel distortion optically without software.

Applications

There are many applications that can benefit from ultra-wide, no distortion lenses. For machine vision, robotics, and automation applications, rectilinear lenses can:

• Aid in object position mapping, increasing situational awareness vital to real-time navigation needs of automated mobile robots and un-crewed vehicles. By providing ultra-wide, undistorted fields of view, rectilinear lens technology enables vehicles to more accurately detect and calculate position to objects to avoid obstacles, ultimately leading to safer and more efficient navigation.
• Enable close focus and wide coverage at the effector end of a robot without work surface blind spots in pick-and-place applications for identification and selection of objects based on color, shape, size, barcode, etc.
• Provide real-time object tracking without the delay or latency inherent to correcting barrel or fisheye distortion. Wide angle coverage of the work surface to improve timing of object positioning and accuracy of robot action.
• Assist measurement and inspection of items in wide areas from close distances, including identifying different types of packages, identifying packaging damage, etc.
• Monitor remote pipelines with high resolution in remote areas. The wide field of view gives good awareness without the need for multiple cameras.
• Human motion analysis using a portable/movable setup. The cameras can be set up in any suitable customer environment and rectilinear lenses allow fast and accurate calibration of the system by removing the need for image de-warping before stitching images together.

Conclusion

Modern vision systems need high performance, wide angle lenses that can display high resolution images of large areas in real time. Fisheye style lenses compress the image, losing resolution to create barrel-distorted images which require software image correction, resulting in latency. Rectilinear lenses correct distortion optically in the lens without software and its related latency, and without loss of resolution due to compression, an elegant and efficient solution.