Processing thermal images

Introduction to thermal mapping

Infrared imaging is increasingly used for obtaining thermal maps, in particular of industrial installations, to quickly detect anomalies in plants, and to better target maintenance efforts and improve the efficiency of operation.

Thermal cameras are rather different from normal RGB cameras. First of all, thermal cameras tend to have much lower resolution than current RGB cameras. They also need special optics, not to block longwave infrared wavelengths. Then, even if thermal cameras normally carry a shutter, this is usually not used for taking pictures, but only for internal calibration of the sensor.

The time over which an image is acquired is rather determined by a “response time” of the camera sensor, which is generally longer than typical exposure times for RGB cameras. In addition, the response of thermal cameras tends to change in time (drift), and be non-uniform over the sensor.

Which images can be used for thermal projects?

Pix4Dmapper can process thermal images that have been captured following the recommendations described in this section.

Format Description
RJPG An RJPG (radiometric JPG) image is a .jpg image with radiometric data embedded in the image's metadata. This is a proprietary image format that is supported by Pix4Dmapper. RJPG is the recommended image format for thermal images.
.tiff .tiff grayscale images are supported by Pix4Dmapper but can lack important radiometric data.
.jpg Pix4Dmapper supports .jpg thermal images, but this image format is not recommended. The .jpg images are colored-mapped temperatures and contain only a visual representation of the temperature instead of the raw values.
 
Information: You can process thermal images extracted from a video but it is not recommended for the same reasons why .tiff images are preferable to .jpg images. Moreover, a movie is less likely to contain image geolocation and is likely to be more compressed than a still image. A thermal video may also capture excessive amounts of image overlap, which can introduce noise into your reconstruction.

Which thermal cameras are supported and which temperature they provide?

Sensor recommendations

In order to have enough visual content in the images for Pix4Dmapper to reconstruct the scene, we recommend:

  • A minimum sensor resolution of 640x480. Smaller sensors are not supported and typically do not calibrate. 
  • Using a lens with a smaller focal length (9mm) increases the image's footprint, though it is possible to use longer lens focal lengths.

Supported thermal cameras

Recommended integrated solutions that are supported out-of-the-box include the following cameras.

Camera model Absolute temperature Relative temperature
DJI Zenmuse XT    x
DJI Zenmuse XTR x  
FLIR Vue Pro   x
FLIR Vue Pro R x  
senseFly ThermoMAP x  
Aeryon Labs FLIR board   x
Workswell WIRIS 2nd Gen 640  x  
Micasense Altum (Multispectral+Thermal)  x  
senseFly Duet T (RGB+Thermal)  x  

Other custom camera integrations based on FLIR's Vue Pro or Tau2 sensors are also supported. Learn more about custom camera integrations: Processing thermal images. For more information on how to process Duet-T images, please have a look at this senseFly article.

 
Important: When creating a project with one of these supported cameras, verify that the Pixel Size and the Focal Length values are correct. Contact your camera manufacturer for details about your camera's specifications. Learn how to modify camera model parameters: How to use the Editing Camera Model Options.

Radiometric thermal cameras

Cameras labeled "R" are radiometrically calibrated. Using such cameras enables the capture of absolute temperature in every pixel of an image. FLIR Vue Pro R and Zenmuse XTR are both radiometric versions that do record absolute temperature. They save their images in RJPG (radiometric JPG) format: a .jpg image with temperature data embedded in every pixel.

 
Important: Even when using radiometrically calibrated cameras, please note that the differences in illumination, as well as the properties of the surface (material, roughness,...) in the scene, also affect the thermal emissivity in such a way that only similar surfaces can be reliably compared within one project. How far the absolute value of temperature is from the locally measured one depends on the details of the camera. Any such bias cannot be compensated by the software alone, but should rather be addressed at the hardware level. In practice, using as reference a local temperature measurement on the ground is the simplest and most effective solution.

How to capture thermal images?

For a better reconstruction of the captured scene in a thermal project, some recommendations should be followed during the image acquisition:

  • Have very high overlap: 90% front and side image overlap.
  • The images have been taken at a resolution of at least 640x480.
  • The images do not suffer from motion blur. An increased flight speed may cause a blurred image.

How to process thermal datasets?

Process a thermal project

1. Create a new project. For more information: New project in Pix4Dmapper.
For nadir datasets with accurate image geolocation, select the processing template Thermal Camera. For more information: Processing Options Default Templates

 
Important: You can apply a different processing template in the following cases:

2. Ensure that the Pixel Size and the Focal Length values are correctly set: on the menu bar, click Project > Image Properties Editor... and in the section Selected Camera Model, click Edit... For step-by-step instructions about how to modify the camera model: How to use the Editing Camera Model Options.
3. On the Processing bar, click Start to start the processing. The thermal index map will be generated during step 3. DSM, Orthomosaic, and Index.

Process dataset from Micasense Altum

The Micasense Altum is a camera with 6 bands: Blue, green, red, red edge, near-infrared (NIR) and thermal infrared (LWIR). Though the thermal sensor is 160x120, the images process successfully as a rig due to the high-resolution multispectral sensors.

1. Upload the images and use the ag multispectral template.
2. To convert the LWIR pixel values to degree C, use the formula: Thermal_ir=(lwir/100)-273.15

There is a demo dataset of this camera in the micasense website.

Process dataset with both thermal and RGB imagery (A better 3D mesh/ model)

Thermal cameras usually have much lower resolution than RGB cameras, and thus the 3D model is of much lower quality. The idea is to use the higher resolution RGB images to compute a detailed 3D model (mesh) and to project the thermal texture on top of it. This greatly improves the final thermal 3D model. To process a dataset with both thermal and RGB imagery:

1. Run step 1. Initial Processing for the thermal dataset following the instructions above.
2. Run step 1. Initial Processing for the RGB dataset in a separate Pix4D project.
3. Merge the RGB and the thermal projects. For more information about merging project: Merging projects.
4. On the menu bar, click Process > ProcessingOptions. Select 2. Point Cloud and Mesh and the tab Advanced. Ensure that for the Point Cloud and Mesh Geometry image groups, Thermal IR is unchecked and that group1 is checked. Ensure that for the Mesh Texture image group, Thermal IR is checked and group1 is unchecked. For more information: Menu Process > Processing Options... > 2. Point Cloud and Mesh > Advanced.

 
Information: This last step ensures that the point cloud will be generated from the RGB images, the mesh geometry will be created from this RGB point cloud and that the texturing will be done using the thermal imagery. However, the merging benefits only the mesh and not any other output.
  

How to visualize thermal outputs?

Visualize the 3D Point Cloud in the rayCloud

1. Click View > rayCloud, to open the rayCloud and load the 3D Point Cloud by ticking the Point Clouds box in the Layers section of the left sidebar. For more information: Menu View > rayCloud > Left sidebar > Layers > Point Clouds .
2. Display: (optional) In the Point Clouds layer of the left sidebar, select Display Properties and change the Shader to either Screen Aligned Quads, Thermal or Spherical Points, Thermal.

Visualize the 3D Textured Mesh in the rayCloud

1. If you use the Thermal Camera processing template and that the 3D Textured Mesh output is desired, on the menu bar, click Process > Generate 3D Textured Mesh. For more information: Menu Process > Generate 3D Textured Mesh.
2. Click View > rayCloud, to open the rayCloud and load the 3D Textured Mesh by ticking the Triangle Mesh box in the Layers section of the left sidebar. For more information: Menu View > rayCloud > Left sidebar > Layers > Triangle Meshes .
3. Display: (optional) In the Triangle Meshes layer of the left sidebar, select Display Properties and change the Shader to Thermal.

Visualize thermal Index Map in the Index Calculator

1. Click View > Index Calculator to open the Index Calculator.
2. In the section Index Map of the sidebar, select the band containing thermal data.

 
Note: Depending on the camera used to capture images, the band containing thermal data will differ.
Zenmuse XTR and FLIR Vue Pro R For projects created with Zenmuse XTR or FLIR Vue Pro R radiometric cameras and if RJPG (radiometric .jpg) imagery is used, the absolute temperature is obtained directly from the band. For more information on RJPG, see this section.
senseFly ThermoMAP Records absolute temperature. For projects created using the senseFly ThermoMAP camera, the temperature [°C or °F] index should be used to obtain absolute temperature index maps. This index is loaded automatically for Thermomap projects and computed using the following formula: 0.01*thermal_ir - 100
Zenmuse XT and FLIR Vue Pro Relative temperature is computed.
Workswell WIRIS 2nd Gen 640 The newer Wiris camera records relative temperature. It is recommended to do the processing using grayscale .tiff images and create the following index to view absolute temperature: 0.04*thermal_ir - 273.15
Micasense Altum LWIR .tiff images are used to generate the thermal reflectance map. Create the following index to view absolute temperature: Thermal_ir=(lwir/100)-273.15

3. Display: (optional) In the section Color Map and Prescription of the sidebar, increase the number of classes to 32 and from the drop-down list choose Equal SpacingMenu View > Index Calculator > Sidebar > 4. Color Maps and Prescription.

How to fix the discontinuities of thermal intensity between consecutive images?

If the temperature seems to drift with time, this is due to the characteristics of the camera (usually uncooled cameras exhibit this behavior) and this cannot be corrected by software. The camera provides an automated way to recalibrate the intensity, usually by taking a picture with the shutter closed. What happens is that the thermal image of a surface having uniform temperature is not itself uniform: it might rather show patterns, peculiar of a specific camera, and highly variable in time.

Check with the camera manufacturer for more details.

How to reduce very long processing time?

There are two main factors that affect the speed of step 1. Initial Processing:

  • Too much overlap: if some images in the project are taken from the same location, this will increase the processing time exponentially. It is advised to use a flight planning app (such as Pix4Dcapture) that triggers the camera based on distance instead of time. Alternatively, it is recommended to manually remove images if the drone was hovering at the same location for an extended period of time.
  • Camera model optimization: if the camera’s initial values are too different from the optimized ones, it may slow down processing. Ensure that the pixel size and focal length are entered correctly: How to use the Editing Camera Model Options.

What to do in presence of entirely white or black images in the rayCloud and low calibration rate?

In case the rayCloud presents either entirely black or entirely white images and the project shows a very low calibration rate, it means that the thermal camera used is not registered in our database. In these situations two actions are possible:

  • The preferred way is to send us a sample of the dataset such that we can include it in our database.
  • Another way is, before starting the process, to close the project and open the .p4d file with a text editor and below the <tangentialT2> and above the <cameraModelSource> lines, add the following line
    <pixelValue pixelType="uint16" min="-1" max="-1"/>

The "pixelType" must match the datatype of your input image. For example, if you use float or 8-bit data, the above line will not work.

How to use a custom integration of a thermal sensor?

When using a custom integration, it is necessary to integrate the metadata Pix4Dmapper requires in the image EXIF tags. Ensure to follow this document listing all the EXIF tags read by Pix4Dmapper: EXIF and XMP tag information read by Pix4D Desktop.

Guidelines for troubleshooting

When the processing of your thermal dataset is not successful or does not calibrate, please ensure to check the following points:

If after verifying the points above, the project still does not calibrate or is still very distorted, please try applying successively the following processing options:

  • Apply All Prior to the internal camera parameter optimization method. For more information on All Prior processing option: Menu Process > Processing Options... > 1. Initial Processing > Calibration.
  • Set the Camera model with distortion parameters to zero (Radials R1, R2, R3, and Tangentials T1 and T2). For more information on camera model options: How to use the Editing Camera Model Options.
  • If it still does not calibrate, try to run it with other calibration methods (Standard, Alternative). For more information: Menu Process > Processing Options... > 1. Initial Processing > Calibration. Thermal and Thermomap template uses alternative calibration pipeline. This pipeline assumes that the dataset does not contain oblique images, the terrain is flat and homogenous. Due to this assumption during the calibration, the images that have  >35 deg orientation will not get calibrated. Try processing with standard calibration.
  • A good thermal data set would be of very high overlap of around 95%. Still, it is important that the images are not captured from the same point of view, but the centers of the images are different points. For cameras like Tau2 (video camera), the fps might be too high and there might be many images in the same location, especially if the drone is hovering. In this case, you should manually remove some frames of the data set.
Was this article helpful?
16 out of 22 found this helpful

Article feedback (for troubleshooting, post here)

82 comments

  • Avatar
    Momtanu (Pix4D)

    Elizabeth, The pixel size for RGB is 1.85 micron and that for thermal is 17 micron. Other parameters are read from EXIF and should be okay. You would only need to edit the pixel size. This link might help: https://community.pix4d.com/t/dji-zenmuse-xt2-calibrated-camera-rgb-and-thermal-19mm/8958

    Edited by Momtanu (Pix4D)
  • karim oueslati

    je veux savoir comment faire dans ce cas pour calculer la température, quelle est la formule adéquate dans le cas dune camera Zenmuse XT 

  • Avatar
    Momtanu (Pix4D)

    Hi, You will need to reach out to the manufacturer to get the formula

  • karim oueslati

    bonjour, 

    je sais pas pourquoi certains images sont non calibrées, meme si je veux utiliser loption calibrage standard dans les option du processus je la trouve pas, avez vous une idee comment faire dans ce cas  

  • Avatar
    Momtanu (Pix4D)

    Hi, You will need to enable advanced to get all the processing options as shown below:

     

    Let us know how it works. If it does not, please send us the logfile and quality report.

  • Miguel Toril

    Hello everyone;

    I followed the steps of this article until the point of processing the dataset of the merged process. In the Point Cloud and Mesh window of the Processing Optins, I cannot select Thermal IR as the group of images to use for the Mesh Texture construction. The program only recognizes "group 1". See attached image. 

    Any idea why? Thanks in advance. 

  • Avatar
    Momtanu (Pix4D)

    Miguel, It means you have jpg images for your thermal project (3 bands, red, green, and blue). The images are not grayscale (rjpg/tif). By default, the images that have the same number and type of bands as well as the images with the same pixel type (byte, float) will be grouped as group1.

  • Quatrailes desairs

    Hello,

    Thank you for this comment box which allows you to understand certain problem by reading it.

    I perform thermal orthophotography of solar panels using a FLIR XT2 on a drone.

    The orthophoto exports are visually consistent but when I run them in Qgis to get temperature information, I have "hot spots" which do not have the same measurement as on Flir tools. For example, I have 35 ° hot spots when I measure the orhopoho on Qgis and 55 ° in the original Flir thermogram.

    Following your previous recomendation, I changed the optimization parameter of the internal camera to "all prior", "use geometrical verified matching" and "custom image scale" on 2. This allowed me to get a little closer more with a hot spot on Qgis at 45 instead of 55 on Flir.

    However, I only have this result with the mosaic (in the 3_dsm_ortho> 2_mosaic renderings) and not with the reflectance map (4_index> reflectance) or the grayscale index (4_index> indices> grayscale) which it always indicates 35 °. Do you know why there is this difference between mosaic and reflectance map? (Both radiometric tiffs)

    Can you help me make a thermal orthophoto that has the original measurements of the raw thermograms?

    Is it also possible to solve this problem by changing parameters in the index calculator?

    If not, should I change the mns parameter by unchecking "filter noise" and "smooth surfaces"?

    I'm exporting to test but I'm on my 15th and I can't find the solution .. Thank you in advance for your help! :)

  • Avatar
    Momtanu (Pix4D)

    Hi, The final reflectance map/ orthomosaic pixel value is calculated as a weighted average of all the pixels in overlapping images. For thermal images, sometimes, you will find that from one image to the next, there is a big change in temperature value though it is the same object/pixel. If the difference is huge, I would recommend contacting the manufacturer and getting the camera calibrated. FLIR recommends doing this once a year to get correct values.

    Reflectance maps are true values whereas orthomosaics are color balanced, thus you will get different values (https://support.pix4d.com/hc/en-us/articles/202739409-Reflectance-map-vs-orthomosaic). 

    You can look at your images one by one in QGIS and you will see the pixel values are changing in your case for the same object.

    Edited by Momtanu (Pix4D)
  • Quatrailes desairs

    Hello, thank you for your answer. Indeed, the values fluctuate depending on the positioning of the thermal fault on my themogram. When the fault is in the center, the temperature is higher while when it is on the sides of the thermogram, the temperature is lower.

    You talk about color balancing on the orthomosaic but the information is thermal and not colrimetric (RGB). The average of the orthomosaic is therefore a function of temperature, isn't it? In this case what is the difference between the average made by the orthomosaic and that made by the reflectance map?

    Thank you in advance,

  • Avatar
    Momtanu (Pix4D)

    You are right. the color balancing is done on the orthomosaic based on the temperature values. In the Reflectance map, there's no color balancing at all, its only a weighted average of the temperature values (same pixel in all overlapping/common images)

  • Quatrailes desairs

    It is noted, thank you for these clarifications. Our camera has a large temperature difference between the center and the edge of the image. This creates a big difference between the reflectance map and the reality. On the other hand, the average of the orthomoisaic allows to have values closer to reality. We will therefore be using orthomosaic for our measurements in the future. Thank you and happy new year !

  • Irene Borra Serrano

    Hi,
    Since the last update we encountered problems to stitch the thermal imagery (WIRIS). 
    Do you have an idea why it could be? Any recommendation? Maybe we are missing a "new" step? Thank you

  • Avatar
    Momtanu (Pix4D)

    Hi Irene, Can you let us know about the problems in detail?  Is the dataset processing successfully in 4.5.6? We did not change anything to processing. It would be great if you can test in 4.5.6, you can download it from here: https://cloud.pix4d.com/download/. Let us know if it works. We would need your p4d file, logfile, and quality report for both versions. 

  • Irene Borra Serrano

    Hi Momtanu,
    We are using the version 4.5.6.

    We can observe areas with lower resolution and holes
    Here the files you asked, thanks for your help
    https://drive.google.com/drive/folders/1f4Qr2SwXqBCcgjLXLd4mbEcWlzJlOB5s?usp=sharing

  • Avatar
    Momtanu (Pix4D)

    Hi, The camera optimization % seems very high which means the camera parameters might be incorrect. Can you try finding the specs and input the correct ones (mainly pixel size in micron) in the image properties editor? After editing the parameters (using the correct ones), or making sure that the parameters are correct, you can use all prior as internal parameters optimization. Let me know how it goes. Also, what was the overlap and flight height?

     

    Edited by Momtanu (Pix4D)
  • Miguel Toril

    Hello everyone;

    in order to test the best way to capture and process thermal images I have flown the same area in two different ways: Nadir and Oblique angle of the camera over the same objects. Prior to that, I have taken standard RGB pictures of the place in order to merge both projects following this tutorial.

    Merging the thermal images with the RGB images produces a reasonable good thermal orthomosaic as you can see in the following picture.

    On the other hand, the merged project which includes the oblique images is not very good and includes many errors as displayed in the example I include here. 

    I would like to improve the quality of the processing with the oblique images, but before making uncountable changes in the setting, I prefer to ask for some advice about where I can start best working on. 

    Thanks in advance.

  • Enrica Marotta

    Dear pix4d team,

    we are going to use your software to create thermal mosaics over volcanic areas with drones and a FLIR VUE PRO. As we use such data for scientific purposes we need to know the error on the temperature introduced by the merging procedure for each pixel in the output. Is it already known? Is there a way to obtain such an information?

    Can you help us?

    Thanks, Enrica

  • Avatar
    Momtanu (Pix4D)

    Miguel Toril I would recommend using "standard" calibration and enable geometrically verified matching for the oblique dataset. Then process step 1. The template optimized for thermal processing (thermal template) has "alternative" calibration pipeline which assumes the images are nadir. Thus we suggest always using nadir images.

  • Avatar
    Momtanu (Pix4D)

    Enrica Marotta Pix4D will depend on the raw images. Thus there will not be any error introduced from Pix4D. Pix4D uses weighted average for calculating the pixel value (temperature in this case) of the final map. The overlapping images containing that pixel are used for this calculation. So whatever error is there in the images will get transferred to the final map.

  • Miguel Toril

    Hello @... and thank you for your reply and your comments. I will process again the images using the settings you mention, but regarding the template I should use, would you recommend to use for this processing with oblique images the 3D Maps template instead of the thermal template?

    I understand it is better to process thermal images with the template designed for that and, as you can see in my previous post, we are already capturing nadir images; nevertheless I still wants to work on the possibility to fly and take images perpendicular to the panels surfaces.

  • Avatar
    Momtanu (Pix4D)

    Hi Miguel, Sorry for such a late reply. I missed this one. For oblique images, you have to use standard calibration. I would recommend still using the thermal template but just changing the calibration mode in step 1. This is because, in 3d maps template, some outputs are disabled. So it would be easier to use thermal and just tweak one parameter. Let me know how it goes.

Article is closed for comments.