Thermal Imaging Processing Technique

Thermal Imaging Processing Technique Provides High Resolution

QVGA (320 x 240) Display

Tetsuo Tamura, Masaharu Imazato, Minoru Akiyama, Toru Tabuchi, Daisuke Hanyu,

Atsushi Machida, Yasushi Uda

Abstract

Applications of Infrared Thermography have been expanded through lower pricing, however, in order to diagnose defects of equipment in building inspection, facility inspection and so on more correctly, low noise and high spatial resolution of Infrared Cameras are needed. We have improved the sensitivity of UFPA (Uncooled Focal Plane Array) and have developed new image processing techniques to obtain higher spatial resolution and better NETD (Noise Equivalent Temperature Difference). We have also developed an affordable QVGA Infrared Camera equipped with these features. We would like to present you the results in this presentation.

Background

During recent years, Infrared Cameras has been widely used in various fields because of their higher performance and lower price. Also, various types of products such as portable cameras for PPM (Predictive Preventive Maintenance), built-in optical filter models for special measurement and fixed mount models for process control have become easily available on the market.

Infrared Thermography is used in non-destructive diagnosis of equipment, in building inspection and facility inspection for PPM. In this field, further improvement of basic performances such as NETD and Spatial Resolution are required for accurate diagnosis and helpful in finding small defects.

We have developed a higher sensitivity of the UFPA (Uncooled Focal Plane Array) by better thermal isolation structure using the MEMS process (Micro Electro Mechanical System). Furthermore we have developed an image processing technique for reducing noise, high spatial resolution and panoramic composition. We hereby report our work with Infrared Thermography and PC Software using these techniques.

Improvement of the Basic performance of the Infrared Camera

Infrared Thermography is used for building inspection and facility inspection where those measured objects are at room temperature, so an infrared range of 8 to 13um wavelength is selected. Further, due to the requirement of low cost and extended battery life, the UFPA is used for the infrared detector. Fig-1 shows the configuration of Infrared Thermography used this UFPA. The basic performance of Infrared Thermography such as NETD and Spatial Resolution are determined by UFPA as well as Infrared Signal Processing and Image Processing. The following is a description of the improvements of each important element.

Improvement of NETD

NETD can be improved by increasing the sensitivity of the detector and by reducing the noise come from the detector of the ROIC (Read-Out Integrated Circuit) and system including bias circuits
applied to UFPA.

Improving the Sensitivity of UFPA

In order to make the sensitivity higher you need to separate (thermally insulate) the light-receiving part of detector more from the external parts. Better isolation produces a bigger change of temperature caused by change of infrared incident on the detector. Thermal isolation is made by a micro-bridge structure (Fig. 2) and using vacuum packaging in order to provide thermal insulation from the environment. Also TEC (Thermo Electric Cooler) is used to control temperature of the detector in order to stabilize output of UFPA and keep the sensitivity high.

Process of MEMS (Micro Electro Mechanical System)

By improving the process of MEMS:

a. Improved thermal isolation of light-receiving part of the detector from ROIC by making the “Legs” of micro-bridge structure that suspends the light-receiving part thinner.

b. Improved Fill-factor by installing shades on the light-receiving part (ratio of effective light-receiving area of detector against detector element.

c. Realized high sensitivity (low NETD) by making ROIC high performance/low noise.(1)

d. Further NEC Guidance and Electro-Optics Division has achieved a NETD of 30mK by reducing the thermal conductance of “Legs” which separate heat from ROIC to light-receiving part of the detector element.(1)

Noise Reduction (Digital Image Processing)

As a way to eliminate noise from infrared image, it is typical that multiple frames of image data are processed with moving average. Temporary noise is reduced to the (number of frames)^-1/2 and the image quality is therefore improved. However when the measured object is moving, motion blur (ghosting) occurs. (Fig. 3-2 Frame averaging image of moving object)

Noise Reduction Technique developed by “AVIO” achieved “Frame Averaging” without ghosting while judging if the measured object is moving or standing still. If a still object, “Frame Averaging” is applied on correspond pixels, but on pixels of moving object, “Frame Averaging” with De-Ghosting is applied. (Fig. 3-3 Frame Averaging with De-ghosting).

The NETD of the original image (Fig. 3-1) is 30mK, the Frame Averaging with de-ghosting is 20mK. The noise was compressed by 0.7 times without any degrade of image, such as motion blur and ghosting.

Improvement of Spatial Resolution

The spatial resolution must be high to measure small objects (related to the size of object and the distance to object) or to find a small defect. If the inspection object is wide or high, it is necessary to measure a wider angle. On the other hand, the number of pixels on the infrared detector is limited, therefore if you measure with one frame, the spatial resolution gets worse. The F.O.V. (Field of View) and resolution has the following relation.

Θx(FOV)　= δ(Spatial Resolution)  x  n(Number of Image Pixels)

Ex. Resolution δ = 1.2mrad, in case of Infrared Thermograph of 320 x 240 pixels, FOV is 22˚ x 16.5˚

This super resolution technique is effective in getting higher resolution while keeping the same FOV. The panoramic function (that is taking multiple images and splicing them together) is effective in expanding the FOV while maintaining the same resolution. We introduced Infrared Thermography Software which was developed in collaboration with AVIO Information Media Processing Laboratory as follows. 

High Resolution with Super Resolution Technique

Our super resolution method creates a high resolution image by using multiple images. It extracts characteristic points from each frame, relates them, matches the position and overlaps them. The overlap also gives the effect of noise reduction.　(Fig. 4-1, 4-2)

High Resolution by Panoramic Composition

When you take a Panoramic Infrared Thermal image, you shoot multiple frames. Those multiple frames are taken with certain overlapped areas or angles. The Infrared Camera is equipped with a gyroscope to keep a certain angle of overlap and make a panoramic image by combining images automatically. The method here is to combine images from each frame smoothly by extracting characteristic points of each frame and adjusting the overlap position to match each point. (Fig. 5)

Examples of Applications for using a High Performance Infrared Imager

Image improved NETD

Fig 4-1 Before Super High Resolution	Fig 4-2 After Super High Resolution

Conclusion

Infrared Cameras can create an image under complete darkness and temperature distribution that cannot be done with a visible camera. In recent years, the prices of Thermal Imagers have become very affordable and the application fields are getting wider. In some applications such as fire detection, security and vision enhancement on airplanes, ships and vehicles, it is used as a replacement for visible cameras due to its features. Therefore it will continue to grow more popular as a commodity product. The key factors to drive these applications are COST and PERFORMANCE of the infrared detector, optics and the image processing

We reported:

We have improved overall detector performance. The higher sensitivity detector makes the cost of the lens lower.

We introduced the Image processing technique to produce a better NETD.

We obtain high spatial images by the super resolution technique and the panoramic images are effective and helpful to diagnose wide-area problems.

In addition, the image processing functions such as optimizing contrast/brightness, target tracking, automatic target recognition and so on, will offer new solutions that help you to resolve issues in your applications.

We will continue to develop new methods and algorithms so the imaging can become more accurate and precise with less power demand from the digital image processor. Incorporating these improvements, the application of Infrared Thermography will expand and the market will grow exponentially.

References:
1) Detection of terahertz radiation from quantum cascade laser using vanadium oxide microbolometer focal plane array. Naoki Oda, etc 、SPIE_ Defense + Security Conf. 6940

2）A. Iketani, T. Sato, S. Ikeda, M. Kanbara, N. Nakajima, and N. Yokoya:

"Super-resolved video mosaicing for documents based on extrinsic camera parameter estimation",

Proc. Asian Conf. on Computer Vision (ACCV2006), Vol. II, pp. 101-110, Jan. 2006

Nippon Avionics Co., Ltd.

URL http://www.avio.co.jp/english/