IN THIS ARTICLE
Introduction to thermal mapping
Which images can be used for thermal projects?
Which Pix4D software can process thermal images?
Which thermal cameras are supported and which temperature?
How to process images obtained from Mavic 3T Enterprise, Mavic 2 Dual Enterprise Advanced, Zenmuse H20T, and Zenmuse H20N?
How to capture thermal images?
How to process thermal datasets?
How to visualize thermal outputs?
How to fix the discontinuities of thermal intensity between consecutive images?
How to reduce very long processing time?
What to do in presence of entirely white or black images in the rayCloud and low calibration rate?
How to use a custom integration of a thermal sensor?
Guidelines for troubleshooting
Introduction to thermal mapping
Infrared imaging is increasingly used for obtaining thermal maps, in particular of industrial installations, to quickly detect anomalies in plants, to better target maintenance efforts, and improve the efficiency of operation.
Thermal cameras are rather different from normal RGB cameras. First of all, thermal cameras tend to have much lower resolution than current RGB cameras. They also need special optics, not to block longwave infrared wavelengths. Then, even if thermal cameras normally carry a shutter, this is usually not used for taking pictures, but only for internal calibration of the sensor.
The time over which an image is acquired is rather determined by a “response time” of the camera sensor, which is generally longer than typical exposure times for RGB cameras. In addition, the response of thermal cameras tends to change in time (drift), and be non-uniform over the sensor.
Which images can be used for thermal projects?
PIX4Dmapper can process thermal images that have been captured following the recommendations described in this section.
| Format | Description |
|---|---|
| RJPG | An RJPG (radiometric JPG) image is a .jpg image with radiometric data embedded in the image's metadata. This is a proprietary image format that is supported by PIX4Dmapper. RJPG is the recommended image format for thermal images. |
| .tiff | .tiff grayscale images are supported by PIX4Dmapper but can lack important radiometric data. |
| .jpg | PIX4Dmapper supports .jpg thermal images, but this image format is not recommended. The .jpg images are colored-mapped temperatures and contain only a visual representation of the temperature instead of the raw values. |
Which Pix4D software can process thermal images?
- PIX4Dmapper and PIX4Dengine can process all types of thermal (RJPG, .tff, .jpg) images that have been captured following the recommendations described in this section.
- PIX4Dfields can process the thermal band of the Micasense Altum and Sentera 6X cameras and will provide real temperature values.
- PIX4Dfields and PIX4Dreact can process the .jpg thermal images of many drones and will provide colored maps that only visually represent the temperature.
- PIX4Dcloud/PIX4Dcloud Advanced alone doesn't support the processing of rjpeg and grayscale thermal images. PIX4Dmapper can be used to process those images. See Processing thermal images using PIX4Dcloud/PIX4Dcloud Advanced and PIX4Dmapper.
- PIX4Dmatic, and PIX4Dinspect do not process thermal images. They support RGB images.
Processing thermal images using PIX4Dcloud/PIX4Dcloud Advanced and PIX4Dmapper.
PIX4Dcloud can be used along with PIX4Dmapper to process the thermal images (rjpeg and grayscale) on the PIX4Dcloud. To do so, first, create a project on PIX4Dmapper and upload it to PIX4Dcloud for processing.
- Create the project using PIX4Dmapper and select the thermal Camera template for processing the images.
- Upload the project to the PIX4Dcloud (Project > Upload project files).
- After processing is completed, download the entire Project files.
- Visualize them using PIX4Dmapper or other third-party software such as QGIS, ArcGIS Pro, etc.
Which thermal cameras are supported and which temperature they provide?
Sensor recommendations
In order to have enough visual content in the images for PIX4Dmapper to reconstruct the scene, we recommend:
- A minimum sensor resolution of 640x480. Smaller sensors are not supported and typically do not calibrate.
- Using a lens with a smaller focal length (9mm) increases the image's footprint, though it is possible to use longer lens focal lengths.
Supported thermal cameras
Recommended integrated solutions that are supported out-of-the-box include the following cameras.
| Camera model | Absolute temperature | Relative temperature |
|---|---|---|
| DJI Zenmuse XT | x | |
| DJI Zenmuse XTR | x | |
| FLIR Vue Pro | x | |
| FLIR Vue Pro R | x | |
| senseFly ThermoMAP | x | |
| Aeryon Labs FLIR board | x | |
| Workswell WIRIS 2nd Gen 640 | x | |
| Micasense Altum (Multispectral+Thermal) | x | |
| senseFly Duet T (RGB+Thermal) | x |
Other custom camera integrations based on FLIR's Vue Pro or Tau2 sensors are also supported. Learn more about custom camera integrations: Processing thermal images. For more information on how to process Duet-T images, please have a look at the senseFly knowledge base (only available with a valid senseFly account).
Radiometric thermal cameras
Cameras labeled "R" are radiometrically calibrated. Using such cameras enables the capture of absolute temperature in every pixel of an image. FLIR Vue Pro R and Zenmuse XTR are both radiometric versions that do record absolute temperature. They save their images in RJPG (radiometric JPG) format: a .jpg image with temperature data embedded in every pixel.
How to process images obtained from Mavic 3T Enterprise, Mavic 2 Dual Enterprise Advanced, Zenmuse H20T, and Zenmuse H20N?
Pix4D's software is currently unable to generate a reflectance map with temperature values from certain thermal cameras, such as the Mavic 3T Enterprise, Mavic 2 Dual Enterprise Advanced, Zenmuse H20T, and Zenmuse H20N from DJI. This is because the metadata of the thermal images from these cameras differs from that of other normal thermal cameras, making it difficult for the FLIR SDK used by Pix4D's software to identify them as rjpeg images. As a result, the software processes these images as RGB images instead of thermal images.
To process these images correctly using Pix4D's software, a workaround is to use third-party software to convert the rjpeg images into TIFF files with temperature information before processing them with PIX4D's software. For more information: Thermal images convertion.
If .jpg thermal images are processed in PIX4Dmapper, PIX4Dfields, and PIX4Dreact, they will be treated as RGB images, and the resulting orthomosaic will not display temperature values. As a result, even if users generate an orthomosaic for visualization purposes, they will not be able to extract the temperature value of a particular object or pixel from the resulting map.
How to capture thermal images?
For a better reconstruction of the captured scene in a thermal project, some recommendations should be followed during the image acquisition:
- Have very high overlap: 90% front and side image overlap.
- The images have been taken at a resolution of at least 640x480.
- The images do not suffer from motion blur. An increased flight speed may cause a blurred image.
How to process thermal datasets?
Process a thermal project
1. Create a new project. For more information: New project in PIX4Dmapper.
For nadir datasets with accurate image geolocation, select the processing template Thermal Camera. For more information: Processing Options Default Templates
- Select processing template ThermoMAP Camera for projects captured with senseFly's ThermoMAP camera. For more information: Processing Options Default Templates
- Select processing template 3D Models for oblique datasets or nadir datasets that do not include image geolocation. For more information: Processing Options Default Templates
2. Ensure that the Pixel Size and the Focal Length values are correctly set: on the menu bar, click Project > Image Properties Editor... and in the section Selected Camera Model, click Edit... For step-by-step instructions about how to modify the camera model: How to use the Editing Camera Model Options.
3. On the Processing bar, click Start to start the processing. The thermal index map will be generated during step 3. DSM, Orthomosaic, and Index.
Process dataset from Micasense Altum
The Micasense Altum is a camera with 6 bands: Blue, green, red, red edge, near-infrared (NIR) and thermal infrared (LWIR). Though the thermal sensor is 160x120, the images process successfully as a rig due to the high-resolution multispectral sensors.
1. Upload the images and use the ag multispectral template.
2. To convert the LWIR pixel values to degree C, use the formula: Thermal_ir=(lwir/100)-273.15
There is a demo dataset of this camera in the micasense website.
Process dataset with both thermal and RGB imagery (A better 3D mesh/ model)
Thermal cameras usually have much lower resolution than RGB cameras, and thus the 3D model is of much lower quality. The idea is to use the higher resolution RGB images to compute a detailed 3D model (mesh) and to project the thermal texture on top of it. This greatly improves the final thermal 3D model. To process a dataset with both thermal and RGB imagery:
1. Run step 1. Initial Processing for the thermal dataset following the instructions above.
2. Run step 1. Initial Processing for the RGB dataset in a separate PIX4Dmapper project.
3. Merge the RGB and the thermal projects. For more information about merging projects: Merging projects.
4. On the menu bar, click Process > Processing Options. Select 2. Point Cloud and Mesh and the tab Advanced. Ensure that for the Point Cloud and Mesh Geometry image groups, Thermal IR is unchecked and that group1 is checked. Ensure that for the Mesh Texture image group, Thermal IR is checked and group1 is unchecked. For more information: Menu Process > Processing Options... > 2. Point Cloud and Mesh > Advanced.
How to visualize thermal outputs?
Visualize the 3D Point Cloud in the rayCloud
1. Click View > rayCloud, to open the rayCloud and load the 3D Point Cloud by ticking the Point Clouds box in the Layers section of the left sidebar. For more information: Menu View > rayCloud > Left sidebar > Layers > Point Clouds .
2. Display: (optional) In the Point Clouds layer of the left sidebar, select Display Properties and change the Shader to either Screen Aligned Quads, Thermal or Spherical Points, Thermal.
Visualize the 3D Textured Mesh in the rayCloud
1. If you use the Thermal Camera processing template and the 3D Textured Mesh output is desired, on the menu bar, click Process > Generate 3D Textured Mesh. For more information: Menu Process > Generate 3D Textured Mesh.
2. Click View > rayCloud, to open the rayCloud and load the 3D Textured Mesh by ticking the Triangle Mesh box in the Layers section of the left sidebar. For more information: Menu View > rayCloud > Left sidebar > Layers > Triangle Meshes .
3. Display: (optional) In the Triangle Meshes layer of the left sidebar, select Display Properties and change the Shader to Thermal.
Visualize thermal Index Map in the Index Calculator
1. Click View > Index Calculator to open the Index Calculator.
2. In the section Index Map of the sidebar, select the band containing thermal data.
| Zenmuse XTR and FLIR Vue Pro R | For projects created with Zenmuse XTR or FLIR Vue Pro R radiometric cameras and if RJPG (radiometric .jpg) imagery is used, the absolute temperature is obtained directly from the band. For more information on RJPG, see this section. |
| senseFly ThermoMAP | Records absolute temperature. For projects created using the senseFly ThermoMAP camera, the temperature [°C or °F] index should be used to obtain absolute temperature index maps. This index is loaded automatically for Thermomap projects and computed using the following formula: 0.01*thermal_ir - 100 |
| Zenmuse XT and FLIR Vue Pro | Relative temperature is computed. |
| Workswell WIRIS 2nd Gen 640 | The newer Wiris camera records relative temperature. It is recommended to do the processing using grayscale .tiff images and create the following index to view absolute temperature: 0.04*thermal_ir - 273.15 |
| Micasense Altum LWIR | .tiff images are used to generate the thermal reflectance map. Create the following index to view absolute temperature: Thermal_ir=(lwir/100)-273.15 |
3. Display: (optional) In the section Color Map and Prescription of the sidebar, increase the number of classes to 32, and from the drop-down list choose Equal Spacing: Menu View > Index Calculator > Sidebar > 4. Color Maps and Prescription.
How to fix the discontinuities of thermal intensity between consecutive images?
If the temperature seems to drift with time, this is due to the characteristics of the camera (usually uncooled cameras exhibit this behavior) and this cannot be corrected by software. The camera provides an automated way to recalibrate the intensity, usually by taking a picture with the shutter closed. What happens is that the thermal image of a surface having uniform temperature is not itself uniform: it might rather show patterns, peculiar of a specific camera, and highly variable in time.
Check with the camera manufacturer for more details.
How to reduce very long processing time?
There are two main factors that affect the speed of step 1. Initial Processing:
- Too much overlap: if some images in the project are taken from the same location, this will increase the processing time exponentially. It is advised to use a flight planning app (such as PIX4Dcapture) that triggers the camera based on distance instead of time. Alternatively, it is recommended to manually remove images if the drone was hovering at the same location for an extended period of time.
- Camera model optimization: if the camera’s initial values are too different from the optimized ones, it may slow down processing. Ensure that the pixel size and focal length are entered correctly: How to use the Editing Camera Model Options.
What to do in presence of entirely white or black images in the rayCloud and low calibration rate?
In case the rayCloud presents either entirely black or entirely white images and the project shows a very low calibration rate, it means that the thermal camera used is not registered in our database. In these situations two actions are possible:
- The preferred way is to send us a sample of the dataset such that we can include it in our database.
- Another way is, before starting the process, to close the project and open the .p4d file with a text editor, and below the
<tangentialT2>and above the<cameraModelSource>lines, add the following line<pixelValue pixelType="uint16" min="-1" max="-1"/>
The "pixelType" must match the datatype of your input image. For example, if you use float or 8-bit data, the above line will not work.
How to use a custom integration of a thermal sensor?
When using a custom integration, it is necessary to integrate the metadata PIX4Dmapper requires in the image EXIF tags. Ensure to follow this document listing all the EXIF tags read by PIX4Dmapper: EXIF and XMP tag information for project creation.
Guidelines for troubleshooting
When the processing of your thermal dataset is not successful or does not calibrate, please ensure to check the following points:
- The images are not too uniform.
- The camera model is correctly set together with the Pixel Size and the Focal Length. Sometimes, the pixel size is not read correctly from the EXIF. This parameter is provided by the manufacturer. For more information on camera model options: How to use the Editing Camera Model Options.
- The image geolocation and orientation are correct without obvious flaws. For more information on image geolocation and orientation: How to select/change the images geolocation and orientation.
- Use Thermal Camera processing template. For more information on processing templates: Processing Options Default Templates.
If after verifying the points above, the project still does not calibrate or is still very distorted, please try applying the following processing options in the following order:
- Apply All Prior to the internal camera parameter optimization method. For more information on All Prior processing option: Menu Process > Processing Options... > 1. Initial Processing > Calibration.
- Set the Camera model with distortion parameters to zero (Radials R1, R2, R3, and Tangentials T1 and T2). For more information on camera model options: How to use the Editing Camera Model Options.
- If it still does not calibrate, try to run it with other calibration methods (Standard, Alternative). For more information: Menu Process > Processing Options... > 1. Initial Processing > Calibration. Thermal and Thermomap templates use THE alternative calibration pipeline. This pipeline assumes that the dataset does not contain oblique images, the terrain is flat and homogenous. Due to this assumption during the calibration, the images that have >35 deg orientation will not get calibrated. Try processing with standard calibration.
- A good thermal data set would be of a very high overlap of around 95%. Still, it is important that the images are not captured from the same point of view, but the centers of the images are different points. For cameras like Tau2 (video camera), the fps might be too high and there might be many images in the same location, especially if the drone is hovering. In this case, you should manually remove some frames of the data set.
having trouble with the prcoesing of Flir vue pro images,temperature difference is present in images.How to solve it?
Hey there. I would suggest opening a support request. Click contact support at the top of the page to open a ticket and include quality reports and a description of the issue, please.
Hi,
I'm planning to do Thermal Image mapping with my DJI Phantom 4 Pro using FLIR VUE PRO Camera. I've few doubts for using this.
From YouTube videos, I understood that both DJI Camera and Flir Vue Pro can be mount at same time and Pix4D is supporting Vue Pro Camera.
Doubt: Can I set Automatic Triggering for Flir Vue Pro in Pix4D Application??
Is that Vue Pro Camera work like DJI Camera without Flir Application??
My purpose is to take Thermal Images using Vue Pro and making 3D Models for Mapping.
Hi Rijun,
Your questions have been answered in the following post.
Don't hesitate to post your questions on our community in a new post or in an already existing post with a similar topic to interact with others users.
Best,
Hello,
Is the Zenmuse XT2 19mm camera supported by Pix4D? thanks
Hello,
The camera is indeed supported.
If you go to the Which thermal cameras are supported section of this article, there is a list including all of the supported cameras and the Zenmuse XT is listed.
Regards.
Indeed the Zenmuse XT is listed, but what about the XT2?
Hi Fredrik,
Currently, the camera Zenmuse XT2 is not included in the camera database of Pix4D mapper. But you can process images from xt2 in Pix4D. It will read the camera parameters from the exif of the images.
Do you have a possible sulotion for this problem? We have tried with both jpeg and rjpg, but get the same error about no calibrated cameras. The overlap should be enough (90%)
Hello,
we have exact the same problem by monitoring fields. There are too many uncalibrated cameras. Overlap was also 90%.
XT2 640@30 HZ on a M210RTK
@Andre @Fredrik, this might be due to oblique images. I am pasting this the article here, guidelines for troubleshooting:
@Fredrik, Your overlap does not seem to be uniform. The orbital and the grid flight path does not have any overlap or common points, as thermal images are of very low resolution. Can you try processing the grid only by removing the images of the orbit? Also, change the calibration to standard.
By changing the calibration method there are more uncalibrated cameras then before....
In the previously calculated model, the images were just not recognized in a "plane", but went into the "depth":
Any ideas ?
Best regards!
Hi Andre,
Since alternative pipeline is optimized for thermal projects, that works the best. The software gives some weight to the geolocation data in this pipeline and since thermal has low resolution and also for agriculture projects or homogeneous areas, using the geolocation data works the best (alternative calibration). Standard pipeline extracts key points and does not use geolocation data while calibration, so that might always be an issue if the area is homogeneous. If you are using standard calibration, if there are oblique images, the project will not fail completely. However, there might be other issues.
It would be best if you write us a support ticket so that we can look into depth through your quality reports, logfile, p4d file and images and work together to solve the issue.
Hi, I wonder if it is possible to modify/adjust the value of the overlapped pixels? For example, I would like to calibrate pixels of the second image based on the pixels of the first during the image stitching process. I assume by default the pixel values on overlapped thermal images in Pix4D are averaged?
Andre, unfortunately, that is not possible. You might try changing the pixels values of the raw images programmatically before processing in Pix4D if you have resources.
Hello, I have a few questions:
Is the DJI Zenmuse XT2 compatible for processing (we want to obtain a thermal map)?
Does the final map have the thermal data and also the geo-reference?
What is the optimal flight height for sensors that have a resolution of 640? What about those that have a resolution of 320?
Thank you.
Hi Alex, yes you can use DJI's Zenmuse XT2 for getting a thermal map. The final map will have temperature values in the pixels and will also be georeferenced if you are not missing the geolocation data for some reason.
The sensor has a resolution of 640x512. Since the resolution is low, we recommend a higher overlap and flight height. You can have a look at this: https://support.pix4d.com/hc/en-us/community/posts/360001400266-500-acre-thermal-orthomosaic and https://support.pix4d.com/hc/en-us/articles/360000173463-Processing-thermal-images#label3
Contribution to the question: What to do in presence of entirely white or black images in the rayCloud and low calibration rate?
In case the rayCloud presents either entirely black or entirely white images and the project shows a very low calibration rate, it means that the thermal camera used is not registered in our database.
Before starting the process, to close the project and open the .p4d file with a text editor and below the
<tangentialT2>and above the<cameraModelSource>lines, add the following line<pixelValue pixelType="uint16" min="-1" max="-1"/>The folders "1_initial" and "temp" (in the proyect) must be deleted before running step 1 "Initial processing"Bye
I have the XT2 19mm 640x512. I need help with several questions regarding the acquisition plan of the thermal capture vs RGB capture..
1. Are there guidelines for ideal altitudes to fly based on the XT2 lens? In other words, if someone has a 9mm lens should they fly at 75', but if I have the 19mm lens I should I fly at 150'?
2. Since the resolution of the thermal camera is much lower than the RGB, are there some guidelines for the AGL of the thermal vs an X4S (or any other sensor)? It would seem the thermal should fly much lower to try and match the image detail of each output.
3. For the acquisition plan (specifically the flight path and gimbal angle) should the thermal be the same as RGB or should the thermal always be at 90° ??
Maybe there is a chart put together that will help with lay out at least a starting point depending on the lens and terrain/structure situation involved. Thanks for any help on this!
Hi Tim,
I will answer the questions below. Users who have had some experience regarding thermal are always welcome to join the conversation :-)
1. Are there guidelines for ideal altitudes to fly based on the XT2 lens? In other words, if someone has a 9mm lens should they fly at 75', but if I have the 19mm lens I should I fly at 150'?
I haven't seen any ideal altitude chart. However, the GSD depends on the focal length.
So with a focal length of 19 mm, you will get a higher resolution at the 150'. To get the similar resolution for the 9 mm lens, you will have to fly lower. The GSD calculator might help: https://support.pix4d.com/hc/en-us/articles/202560249-TOOLS-GSD-calculator and also this link: https://support.pix4d.com/hc/en-us/articles/202558849-How-to-select-Camera-Focal-Length-and-Flight-Altitude-considering-the-desired-Ground-Sampling-Distance-and-Terrain-to-map
2. Since the resolution of the thermal camera is much lower than the RGB, are there some guidelines for the AGL of the thermal vs an X4S (or any other sensor)? It would seem the thermal should fly much lower to try and match the image detail of each output.
For thermal cameras you should actually fly higher to get more keypoints as the resolution in very low. It is like moving a book away from your eyes, you are able to see much clearer and detect features. I have used 60 m flight height and good good results for thermal. So fly at a such a height or higher and both RGB and thermal will work.
3. For the acquisition plan (specifically the flight path and gimbal angle) should the thermal be the same as RGB or should the thermal always be at 90° ??
Thermal template is designed for nadir images. So it is good to have both RGB and thermal nadir unless you want to capture facades of buildings.
Good morning,
Is it possible to detect plot for a thermal image, and how can export the relative temperature data as a csv or an excel file.
Thank you so much.
It is possible to draw regions, but as you keep adding regions the statistics (mean, median etc) gets accumulating. You cannot export as csv though, that would be a cool feature and important. The workflow for adding regions for statistical analysis:
Thank you so much for your reply.
I was wondering if you could show em the workflow to generate the thermal index (to get the the absolute or relative temperature).
Thank you so much.
Francia, If you are using a radiometric camera and rjpeg format, you will get the absolute temperature. If you are using tif, you will get relative temperature. For processing workflow, have a look here: https://support.pix4d.com/hc/en-us/articles/360000173463-Processing-thermal-images#label4.
You have to process with thermal template and at the end of step 3, when you click on index calculator (left sidebar), you will get the temperature map. Hope this helps.
The post thermal project has a new comment.
Momtanu (Pix4D)
Hi Michele,
Thermal_ir and grayscale is the same and you will get the same results.Check grayscale and process only step 3. The warnings should not cause any failure, are you getting any errors? If the project fails, can you upload the logfile to google drive/onedrive and send me the link?
View the comment
Momtanu,
I can't see my post and your comment anymore so I'm writing to you here.
The project seems to work even if step 3 seems not completed (in red at the end of the process - image attached).
I am attaching the logfile anyway. Here is the link:
https://1drv.ms/u/s!AhPYCzICplBoh-dp2EkKywHwxljUVA?e=NS7aYS
Thank you!
Michele
Dear pix4d team,
I followed the merge procedure (rgb+thermal) to obtain a better result in a thermal ortho mosaic.
At the end I haven't any thermal map.
I use a dji Mavic duo.
Can you help me?
Hi Pix4d team and XT2 users,
Does any one of you know the formula for computing temperature from the Zenmuse XT2 RJPG images?
thanks!
@Kang, If the images are rjpeg, the temperature values will be absolute, you do not need any conversion formula. Are you getting values outside your expected range?
@Hydro lab, Can you send us your merged quality report and p4d file? You can upload in google drive and send us the link here or you could also write us a support ticket.
@Michele I just saw your message today. I apologize. Did you already figure out what's going on? Also, what happens when you click on the index calculator? are you able to generate the reflectance (temperature) map?