Masks - PIX4Dmatic

This article explains in detail the Masks feature in PIX4Dmatic.

What is a Mask?

A mask is a binary segmentation of an image, used to select specific pixels that belong to a particular object or region, such as a car or a pipe in a trench. Masks are automatically generated based on prompts, which are significant points manually selected to indicate inclusion or exclusion from the mask. A group of masks across different images forms a coherent Mask object, represented as a circle in the 3D viewer and listed in the currently active layer.

The image above shows 3 different masks with different colors (yellow, green, and pink). 

How to create a Mask

1. Add a Layer (optional): Add a Layer in the Content panel. Customize the layer by choosing a color and providing a name. If no layer is created, the first validated mask will create a new layer automatically.
2. Activate Image Mask Tool
3. Start Prompting: When at least one mask has been created and validated a “Mask” entry is added to the active layer. Click Apply to 3D to create a marker in 3D. This marker will be displayed together with the selection of the point cloud. The dimensions of the object will be calculated and added to the property panel.
4. Multiple Viewpoints: To achieve a more accurate 3D selection, create masks from different viewpoints on a sufficient number of images.

How to Create Masks on Images: Prompting

To create masks on images, use prompts:

• Positive Prompts: Single left mouse button clicks to highlight areas to include (represented with green circles).
• Negative Prompts: SHIFT + left mouse button to exclude areas (represented with red circles).
• Validate Mask: When the mask is satisfactory, press the ENTER key to validate it. Note that starting to prompt a new image (or disabling the image mask tool) will automatically validate the previous mask.

Prompts help indicate which parts of a larger object should be included or excluded. In the example below, we aim to select only the front part of the car. After two positive prompts, the selection is too large. By adding a negative prompt, we achieve the desired selection.

Creating Multiple Masks in the Same Image

You can create multiple separate masks within the same image. This is useful for representing disjoint components of a larger object. For instance, to select all the glass parts of a car (windshield and headlights), start with positive prompts. If the selection is too large, press ESC to cancel the last prompt, validate the first mask (headlights) with ENTER, and then create a new separate mask for the windshield.

How to use Masks

Once image masks have been created, the Apply to 3D button can be used to create a selection in 3D, highlighted with a Mask object. The Mask object can be utilized in various ways:

• Measure an Object

The dimensions (length, width, and height) of the 3D object are automatically measured and added to the object's properties when Apply to 3D is performed. An existing Mask can be updated to improve the 3D selection.

• Classify or Clean a Point Cloud

Clicking Apply to 3D with one or more image masks creates a 3D selection, similar to other selection tools. You can refine it with other tools, disable points, or assign an ASPRS class. These actions are available from the context menu. This method offers more refined control compared to the Object Selection Tool.

• Improve Dense Point Cloud and Mesh

Masks can enhance the quality of dense point clouds and meshes by removing selected elements from the scene, effectively hiding them from 3D. Specify the layers to be used in the Dense point cloud and Mesh steps under the Mask-aware setting.

• Improve Texture in a Mesh

Typically, the same mask layers should be used in both dense and mesh creation to completely remove the masked element from the scene. However, you can configure the mesh and dense settings so that the mesh retains the geometry of the masked part, but the corresponding mesh elements are not textured. This is useful for removing artifacts from reflective or transparent surfaces like windows or to reduce artifacts due to thin elements.
For instance, masking the lamp that causes artifacts on the wall textures improves the room's appearance.

FAQ

• What is the difference between the 3D Object Selection Tool and Mask workflows?

The Mask workflow offers more precise control over selection, particularly for objects viewed from multiple angles, such as statues. The 3D Object Selection Tool may not provide satisfactory results in such cases, whereas mask workflows enable precise 3D selections from different viewpoints.

• What is the minimum number of Masks needed to create a 3D Mask marker?

The number of image masks needed depends on their intended use. For documentation, point cloud cleaning, or classification, a single image mask may suffice. For reliable object dimension measurements, multiple viewpoints of the same object are recommended. For dense and mesh applications, create image masks in all images where the object appears.

• I created a Mask marker, then added/edited/deleted more masks, but nothing happened in 3D. Why?

After adding, removing, or changing image masks, you need to click "Apply to 3D" again to apply the changes.

• How can I improve the precision of a Mask?

Use multiple prompts to refine a mask. A single positive prompt (green circle) highlights one component of an object. Add more positive prompts for additional components or use negative prompts to exclude unwanted parts.

• How do I Mask several disjoint objects in the same image (e.g., all windows of a facade)?

Create multiple masks, one for each object. Use positive prompts to select the first object, validate the mask with ENTER, and then proceed to annotate the other objects similarly.

• Why is the quality of the masked Dense point cloud and Mesh not as good as expected?

For best results, ensure that all images where the object appears are masked. This workflow will be simplified in future updates.