Magic Tool - PIX4Dfields

This article introduces the Magic Tool in PIX4Dfields, an Artificial Intelligence (AI) assisted selection tool for quickly detecting and selecting weed nests, damage, and other anomalies in orthomosaics and index layers.

How to access the Magic Tool

Access the tool by clicking the Magic icon next to the Index icon.

[1] Layer selection

• Hide or show annotations that may help when using this tool.
• Select the Source layer where Magic Tool will run on top. 
• Select a Boundary of the area from the Source layer where the Magic Tool will work.
• Select Obstacles if needed. These zones will be discarded by the Magic Tool.

[2] Grid creation

A grid needs to be created for classification purposes when using the Magic Tool. The definition of the grid is based on the following aspects:

• Cell size: Select a cell size that roughly represents the structures that need to be detected on the map. When planning a herbicide spot spraying mission, the cell size could match the implements section or working width.
• Rotation: With the rotation slider, the orientation of the grid can be changed. It is recommended to use the same orientation as the crop-row or the main driving direction.
• Horizontal offset: The grid can be shifted horizontally if needed. The mouse wheel can be used to fine-tune each value in smaller steps.
• Vertical offset: The grid can be shifted vertically if needed. The mouse wheel can be used to fine-tune each value in smaller steps.

Once finished with setting the appropriate values for the grid generation, click Start Magic. It usually takes just a few seconds while the AI learns about the structure of the data.

[3] Labeling

In this step, Wanted and Not wanted cells are labeled in order for the AI to start working.

• At least three Wanted cells need to be marked.
• After selecting the Wanted cells, continue with marking the Not wanted cells, as this is a necessary step. They can be selected via the sidebar or by holding the Shift key to invert the mode temporarily.
• With three labels for each type of cell (Wanted, Not wanted) the Magic Tool will start tracking down similar areas on the whole field instantly.
• More labels can be introduced to train the Magic Tool further and improve its detection performance.
• Once finished with this step, continue with the Cleanup step.

[4] Cleanup

Corrections to the results are made during the Cleanup step:

• Select from the Brush section to Add or Delete cells from the selection and adjust the brush size.
• Hold Crtl key, click and drag the cursor to paint multiple cells in a row.
• Hold Shift + Ctrl to invert the mode.
• The Buffer function allows to add or subtract from the selection systematically.
• There is also the option to click on the revert button, on the top-right side of the menu, to return to the initial selection.

[5] Convert

The detections can be converted to:

• Area annotation: The detections will be saved as area annotations. This is useful when detecting weed nests, crop damage, measuring areas, etc.
• Operation:  The detections will be used to create a Targeted Operation, which will fast-track the process of creating a prescription map. 
  
• Boundary: the detections will be saved as a boundary. This is useful when trying to mark the field boundary by using the Magic tool.
• Obstacle: the detections will be saved as obstacles. This is useful when using the Magic Tool to detect trees that should be avoided in the mission, for example.
• It is recommended that one subpolygon per grid cell be toggled off. Cells will be merged, resulting in simpler geometry.
• Give a descriptive Annotation name.
• Toggle on Learn based on my data to give the Magic Tool the option to learn from your data. This data will be anonymized training data for more magic in the future!
• Click Convert.

What to do with Magic Tool outputs? 

Magic tool outputs can be exported from the Export menu.

When doing spot spraying or variable-rate applications, it is recommended to create a Targeted Operation directly in Step 5, selecting Operation.
If the Magic tool outputs were saved as Annotations, use these Annotations to  create a Targeted Operation:

1. Go to Operations.
2. Select "Use existing annotations to pre-fill the grid".
3. Magic tool output as Boundaries and obstacles can also be used in Targeted Operations

Once Magic Tool outputs have been used in Targeted Operations, the files are ready for use in tractors, ground sprayers, spraying drones, and other equipment.

Troubleshooting

• If the Magic Tool is taking too long to process, reprocess the dataset with a reduced GSD or export the existing layer with the 50% scale option selected and reimport it into the project
• If there is no detection, export the layer as RGB, reimport it into the project, and restart the Magic Tool with it.
• R00018: The annotations have an invalid format. This error could be caused by special characters in the user's name (accent mark, middle dash)