Quick links
      Back to home
      1. Pix4D Documentation
      2. PIX4Dmapper
      3. Quick links

      Quality Report Help - PIX4Dmapper

      Video: Watch the instructional video related to Quality Assessment:
      Important:

      Processing Failed in Step 1

      If processing failed during step 1, the Failed Processing Report displays the following information:

      • Error: Description of the error that made the processing fail.
      • Substep: The substep of Initial Processing at which the processing failed.
      • Cause: Description of the possible causes of the failure.
      • Solution: Description of the possible solutions with a link to step by step instructions.

      Quality Check

      Images

      Keypoints Image Scale > 1/4: More than 10'000 keypoints have been extracted per image.


      Keypoints Image Scale ≤ 1/4: More than 1'000 keypoints have been extracted per image.

      Images have enough visual content to be processed.

      Keypoints Image Scale > 1/4: Between 500 and 10'000 keypoints have been extracted per image.

      Keypoints Image Scale ≤ 1/4: Between 200 and 1'000 keypoints have been extracted per image.

      Not much visual content could be extracted from the images. This may lead to a low number of matches in the images (for more information: Matching Quality Check) and incomplete reconstruction or low quality results. This may occur due to several factors:

      • Image content: Large uniforms areas such as deserts, snow, fog, etc.
        What to do: In such cases, a high overlap is required. Flying at a higher altitude may also have a positive influence on the visual content of the images.
      • Image quality: Images are over/under exposed, blurry, or noisy.
        What to do: Camera parameters need to be adjusted (shutter speed, exposure time). For more information about the camera settings: Step 1. Before Starting a Project > 2. Configuring the Camera Settings.
      • Image size: The likelihood of extracting many features increases with the image size.
        What to do: Images smaller than one megapixel have very few features and require a large amount of overlap (>80%). Doubling the image size used to extract the features could also help: Menu Process > Processing Options... > 1. Initial Processing > General.

      Keypoints Image Scale > 1/4: Less than 500 keypoints have been extracted per image.

      Keypoints Image Scale ≤ 1/4: Less than 200 keypoints have been extracted per image.

      What to do: Same as above, increasing the overlap (> 90%).

      Dataset

      More than 95% of enabled images are calibrated in one block.

      All or almost all images have been calibrated in a single block.

      Between 60% and 95% of enabled images are calibrated

      or

      more than 95% of enabled images are calibrated in multiple blocks.

      Many images have not been calibrated (A) or multiple blocks have been generated (B).

      A) Uncalibrated images are not used for processing. This may occur due to several factors:

      B) Multiple blocks: A block is a set of images that were calibrated together. Multiple blocks indicate that there were not enough matches between the different blocks to provide a global optimization (see the graph 2D Keypoint Matches). The different blocks might not be perfectly georeferenced with respect to each other.

      What to do:

      Less than 60% of enabled images are calibrated.

      What to do: Same as above.
      Such a low score may also indicate a severe problem in:

      • The type of terrain: Water surface, oceans, mirrors and glass surfaces, moving lava, and moving landscapes do not contain the needed visual content for processing. To obtain results, these terrains need to be combined with areas that are easy to reconstruct. Flying at a higher altitude is recommended to map areas close to water: Step 1. Before Starting a Project > 1. Designing the Image Acquisition Plan > a. Selecting the Image Acquisition Plan Type.
      • Image acquisition process: Wrong image geolocation, inappropriate flight plan, insufficient overlap, corrupted images, etc.
      • Project setup: Wrong coordinate system definition, wrong images, etc.

      Camera Optimization

      An initial camera model should be within 5% of the optimized value.

      The focal length/affine transformation parameters are a property of the camera's sensor and optics. It varies with temperature, shocks, altitude, and time. The calibration process starts from an initial camera model and optimizes the parameters. It is normal that the focal length/affine transformation parameters are slightly different for each project. An initial camera model should be within 5% of the optimized value to ensure a fast and robust optimization.

      The percentage of the difference between the initial and optimized focal length for a perspective lens, affine transformation parameters C and F for a fisheye lens, is between 5% and 20%

      What to do:

      • Flat and homogenous areas that do not provide enough visual information for optimal camera calibration.
      • The images have significant rolling shutter distortion.
      • Blurry images, a dataset with low overlap, damaged camera.
        • Recapture Image Dataset
        • Check the camera for any potential damage
      • Wrong initial internal camera parameters. The camera has been physically modified with a lens filter or similar accessory that affects the optic.
      • Preprocessed or cropped images.
        • Do not modify the images before importing into Pix4D for processing.

      The percentage of the difference between the initial and optimized focal length for a perspective lens, affine transformation parameters C and F for a fisheye lens, is more than 20%.

      What to do: same as above.

      Matching

      Keypoints Image Scale > 1/4: More than 1'000 matches have been computed per calibrated image.

      Keypoints Image Scale ≤ 1/4: More than 100 matches have been computed per calibrated image.

      This indicates that the results are likely to be of high quality in the calibrated areas. Figure 5 of the Quality Report is useful to assess the strength and quality of matches.

      Keypoints Image Scale > 1/4: Between 100 and 1'000 matches have been computed per calibrated image.

      Keypoints Image Scale ≤ 1/4: Between 50 and 100 matches have been computed per calibrated image.

      A low number of matches may indicate that the results are not very reliable: changes in the initial camera model parameters or in the set of images may lead to improvements in the results. Figure 5 of the Quality Report shows the areas with very weak matches. A low number of matches is very often related to low overlap between the images.


      What to do: See the Dataset Quality Check section to improve the results. There might be needed to restart the calibration a few times with different settings (camera model, Manual Tie Points) to get more matches. To avoid this situation, it is recommended to acquire images with more systematic overlap. For more information about the flight plan: Step 1. Before Starting a Project > 1. Designing the Image Acquisition Plan > a. Selecting the Image Acquisition Plan Type.

      Keypoints Image Scale > 1/4: Less than 100 matches have been computed per calibrated image.

      Keypoints Image Scale ≤ 1/4: Less than 50 matches have been computed per calibrated image.

      Failed Processing Report: always displayed as the information is not available.

      What to do: Same as above. The minimum number of matches to calibrate an image is 25.

      Georeferencing

      GCPs are used and the GCP
      error is less than 2 times the average GSD.

      For optimal results, GCPs should be well distributed over the dataset area. Optimal accuracy is usually obtained with 5 - 10 GCPs. For more information about GCPs: Step 1. Before Starting a Project > 4. Getting GCPs on the field or through other sources (optional but recommended).

      GCPs are used and the GCP
      error is more than 2 times the average GSD.

      or

      no GCPs are used

      Failed Processing Report: always displayed whether GCPs are used or not.

      A) GCPs are used

      • Not enough GCPs were used given the size of the project.
        • Add additional GCPs. A minimum number of 3 GCPs is required but we recommend including at least 5 to 10 GCPs. When the topography of the area is complex adding more GCPs could lead to a more accurate reconstruction.
      • Accuracy values for GCPs are not properly set.
        • If the accuracy of the GCPs is known then it should be entered instead of the default values in the GCP/MTP Manager.
        • When using the default accuracy values in projects that are set to imperial (ft) then the GCP accuracy should be adjusted to 0.066.
      • Poor GCP marking: The quality of GCP marks can be estimated by reviewing the projection error for each GCP shown in the quality report. All projection error values should be less than 1 pixel.
      • GCPs are not marked on enough images.

      B) No GCPs are used
      There are two cases where No GCPs are displayed:

      • No GCPs were entered. This means that the project is georeferenced using the position of the computed image positions. GPS devices used to geolocate the original images may suffer from a global shift, leading to a global shift in the project of several meters.
      • The GCPs were discarded by the software due to errors with the GCPs (e.g. wrong GCP coordinate system, wrong GCP coordinates, GCPs not marked correctly on the images). For more information: Ground Control Points.

      GCPs are used and the GCP error is more than 4 times the average GSD.

      A GCP error superior to 4 times the Ground Sampling Distance may indicate a severe issue with the dataset or more likely an error when marking or specifying the GCPs.

      What to do: Same as above.

      Preview

      The displayed images are the low resolution preview of the Orthomosaic and the DSM before step 2. They allow a visual inspection of the quality of the initial calibration. If the orthomosaic is skewed, there might be an error with the project orientation and GCPs may be required. If the DSM contains large seams or artifacts, there might be due to multiple blocks in the reconstruction. If there are holes in the Orthomosaic and DSM, check the Quality Checks section and the 2D Keypoint Matches graph.

      Initial Image Positions

      This graph is useful to review the images geolocation. If this graph does not correspond to the flight plan, there might be problems with the matching, and the orientation, scale and/or the geolocation of the results. Check that the images' coordinate system and the coordinates of the images are correct.

      Computed Image/GCPs/Manual Tie Points Positions

      This graph shows the difference between the initial and computed image positions, the difference between the initial and computed GCPs/Check Points positions (if any), the MTPs positions (if any) and the uncertainty ellipses of the absolute camera positions.

      Images: There might be a small offset between the initial and computed image positions because of image geolocation synchronization issues or GPS noise. If the offset is very high for many images, it may affect the quality of the reconstruction and may indicate severe issues with the image geolocation (missing images, wrong coordinate system, and/or coordinate inversions).

      A bended/curved shape in the side and front view may indicate a problem in the camera parameters optimization. Ensure that the correct camera model is used. If the camera parameters are wrong, correct them and reprocess. If they are correct, the camera calibration can be improved by:

      • Increasing overlap/image quality.
      • Removing ambiguous images (shot from the same position, take-off or landing, too much angle, image quality too low).
      • Introducing Ground Control Points.
      • Enable linear shutter optimization in the Image Properties Editor.

      GCPs/Check Points: An offset between initial and computed position may indicate severe issues with the geolocation due to the wrong GCP/Check Point initial positions, wrong coordinate system, and/or coordinate inversions, wrong marks on the images, wrong point accuracy.

      Uncertainty Ellipses: The absolute size of the uncertainty ellipses does not indicate their absolute value because they have been magnified by a constant factor noted in the figure caption. In projects with GCPs, the uncertainty ellipses close to the GCPs should be very small and increase for images further away. This can be improved by distributing the GCPs homogeneously in the project.

      In projects only with image geolocation, all ellipses should be similar in size. Exceptionally large ellipses may indicate calibration problems of a single image or all images in an area of the project. This can be improved by:

      • Adding Manual Tie Points in the area.
      • Rematching and optimizing the project.
      • Removing images of low quality.

      Absolute Camera Position and Orientation Uncertainties

      In projects only with image geolocation, the absolute camera position uncertainty should be similar to the expected GPS accuracy. As all images are positioned with similar accuracy, the sigma reported in the table should be small compared to the mean. In such projects, the absolute camera position uncertainties may be bigger than the relative ones in the table “Relative position and orientation uncertainties”.

      In projects with GCPs, a large sigma can signify that some areas of the project (typically those far away from any GCPs) are less accurately reconstructed and may benefit from additional GCPs.

      Overlap

      This graph shows the number of overlapping images for each pixel of the orthomosaic. It only takes into account the calibrated images. Red areas indicate too low overlap. This may lead to low-quality 3D reconstruction in these areas. The overlap is an important parameter for the overall quality. For precise 3D modeling and mapping applications, the overlap should be in green, i.e. each pixel should be visible in more than 5 images.

      Internal Camera Parameters

      Perspective lens: The principal point should be around half the resolution of the camera, and that the radial distortion values R1, R2, R3 should each be smaller than 1. The uncertainties on the focal length and the principal point should be only a few pixels. The uncertainties on the distortion parameters should be close to zero. For more information about how to edit the initial camera parameters: How to use the Editing Camera Model Options.

      Fisheye lens: The principal point should be around half the resolution of the camera. The optimized values for the affine transformation parameters C and F should be close to each other. The optimized values for the affine transformation parameters D and E should be close to 0. The uncertainties on the affine transformation parameters C, D, E, and F should be only a few pixels. The uncertainties on the polynomial parameters should be close to zero. For more information about how to edit the initial camera parameter: How to use the Editing Camera Model Options.

      For more information: Camera Optimization section in the Quality Check table.

      Parameters Correlation: The highest quality calibration is characterized by decorrelated parameters. However, having some type of correlation between parameters is sometimes expected:

      • In projects with nadir images, a correlation between the radial distortion parameters and between the coordinates of the principal point is expected.
      • In close-range projects with oblique images, a correlation between the focal length and the coordinates of the principal point is expected as well as between the coordinates of the principal point and the tangential distortions.

      What helps with decorrelation:

      • Accurate camera positions (e.g. RTK, or at least GPS), combined with GCPs.
      • MTPs at multiple depths and near image edges for oblique projects.
      • Different camera orientations (e.g. rotate at ends of grid rows, so images will be at 180°), especially when the coordinates of the principal point are correlated.
      • Using All Prior for the internal parameters: Menu Process > Processing Options... > 1. Initial Processing > Calibration.

      2D Keypoints Table - 2D Keypoint Matches graph

      See the Matching and Dataset Quality Check sections in the Quality Check.

      The 2D Keypoints Table displays some statistics of the keypoints and the matches of the project. Keypoints are points of interest (high contrast, interesting texture) on the images that can be easily recognized. The number of keypoints depends on:

      • The size of the images.
      • The visual content.

      A 14MP image will generate between 5'000 and 50'000 keypoints per image. If the number of keypoints is less than 1'000, the image may not have enough content to be calibrated (Images section in the Quality Check). The number of matches will be very low if:

      • The number of keypoints is low.
      • Visual content is too repetitive.
      • The overlap between the images is too low.
      • There are too many changes in the scene during the image acquisition (moving shadows, cars, etc).

      The minimum number of matches to calibrate an image is 25. The recommended number of matches is at least 1'000 per image. In the 2D Keypoint Matches graph, it is possible to visualize the areas with weak matches. It might be necessary to acquire images again in these areas to increase the image overlap.

      The Uncertainty Ellipses describe how precisely each image is located with respect to the other images by means of the Manual and Automatic Tie Points. Usually, the ellipses in the center of the project are smaller than at the outside, as these images have more matches that bind them to the surrounding images. Large ellipses in parts of the project may indicate problems calibrating these parts of the project and typically correspond to areas with few matches.

      Relative Camera Position and Orientation Uncertainties

      The mean relative camera position uncertainty should be within a few multiples of the GSD, the mean orientation uncertainty should be less than 0.1 degrees. A large sigma may indicate that some parts of the project are not well calibrated.

      In projects with RTK-GPS or with many GCPs the relative camera position and orientation uncertainties reported in this table may be worse than the ones reported in the table “Absolute Camera Position and Orientation Uncertainties”. This is expected as this table provides information on how well the Tie Points constrain the images.

      2D Keypoint Table for Camera

      This table is displayed if more than one camera model is used. It displays some statistics of the keypoints and the matches for each camera model. The same analysis as above applies.

      Median / 75% / Maximal Number of Matches Between Camera Models

      This table is displayed if more than one camera model is used. It shows the median, 75% quartile, maximum number of matches between two different camera models and between the images of one camera model. The same analysis as above applies.

      3D Points from 2D Keypoint Matches

      Multiple 2D matching keypoints are triangulated together using the camera parameters to generate a 3D point. 3D points generated from 2-3 images are less precise than 3D points generated from a higher number of images.

      Manual Tie Points

      This section is displayed if MTPs have been used. For a good calibration, Manual Tie Points should have an error around 1 pixel. The Manual Tie Points image marks should also be Verified. High Projection Error and/or many not Verified marks may indicate an issue with the marking or the calibration.

      Ground Control Points

      This section is displayed if GCPs have been used. GCPs are used to assess and correct the georeference of a project. For more information: Georeferencing. 3 GCPs is the minimum to geolocate (scale, orient, position) a project. Optimal accuracy is usually obtained with 5 - 10 GCPs. For more information about GCPs: Step 1. Before Starting a Project > 4. Getting GCPs on the field or through other sources (optional but recommended).

      Scale Constraints

      This table is displayed if the project has Scale Constraints. It displays the Computed Length Error for the scale constraint. Verify that the mean error is close to 0 and the sigma error is close to 1. If not, verify that:

      • The Initial Length accuracy for the scale constraint is correct.
      • The image geolocation accuracy is correct if the images are geolocated.
      • The GCPs' accuracy is correct if the project has GCPs.

      Orientation Constraints

      This table is displayed if the project has Orientation Constraints. It displays the Computed Angular Error in degrees for the orientation constraint. Verify that the mean error is close to 0 and the sigma error is close to 1. If not, verify that:

      • The Angular Accuracy for the orientation constraint is correct.
      • The image geolocation accuracy is correct if the images are geolocated.
      • The GCPs accuracy is correct if the project has GCPs.

      Absolute Geolocation Variance

      This table displays the percentage of geolocated and calibrated images with a geolocation error in X,Y,Z within a predefined error interval. There are ten predefined intervals between -1.5 and 1.5 times the maximum accuracy Amax of all images. If the percentage of images with an error lower than -1.5 × Amax or higher than 1.5 × Amax is big, the Accuracy values might not have been set correctly. Verify if the Accuracy of the image geolocation and the GCPs needs to be adjusted.

      This table also evaluates the quality of the image geolocation. A high percentage of images with a high error may indicate:

      • Noise in the GPS device.
      • Poor synchronization between the GPS device and the camera.
      • Errors in the geotagging process.

      Geolocation Coordinate System Transformation

      This table is displayed if a Site Calibration transformation is defined and enabled and if the output coordinate system is an arbitrary coordinate system. It defines the transformation from the input coordinate system to the output arbitrary coordinate system.

      It can be used in projects where the images are in a known coordinate system and no GCPs are used in order to define the transformation to an arbitrary output coordinate system. For more information: How to compute the Site Calibration for GCPs in an Arbitrary Coordinate System.

      Relative Geolocation Variance

      This table displays the percentage of geolocated and calibrated images with a Relative Geolocation Error between -1 and 1, -2 and 2 and -3 and 3. A high percentage of images with a Relative Geolocation Error lower than -3 or higher than 3 may indicate an incorrect value for the Accuracy of the image geolocation (given by the user). Verify if the Accuracy of the image geolocation and the GCPs needs to be adapted.

      In projects without GCPs:

      • If less than 99.6% of the geolocated and calibrated images have an error between -3 and 3, then the geolocation Accuracy might be overestimated. Try to increase the value of the geolocation Accuracy.

      Rolling Shutter Statistics

      This section is displayed if the Linear Rolling Shutter model has been selected. The graph contains the vector between the initial camera position and the final camera position during the readout time (time needed to take the image). The vector should have the same direction as the flight direction and the Median camera speed should correspond to the drone speed during flight. If this is not the case, there might be a problem with the calibration. Check the number and quality of matches, camera parameters, the overlap of the project.

      The table displays the

      • Median camera speed.
      • Median rolling shutter displacement(during sensor readout).
      • Median rolling shutter time.