This article introduces the Magic Tool in PIX4Dfields, an Artificial Intelligence (AI) assisted selection tool to quickly detect and select weed nests, damage, and other anomalies in orthomosaics and index layers.

 How to access the Magic Tool

Access the tool by clicking the  Magic icon located next to the Index icon.

Magic Tool icon.

 [1] Layer selection 

First of all, the Source layer where Magic Tool will run needs to be selected.

There is always a possibility to show or hide previous annotations that may help when using this tool.

Selecting the Source layer from the Magic Tool menu.

 [2] Grid creation

A grid needs to be created for classification purposes when using the Magic Tool. The definition of the grid is based on the following aspects:

• Cell size: Select a cell size that roughly represents the structures that need to be detected on the map. When planning a herbicide spot spraying mission, the cell size could match the implements section or working width.
• Rotation: With the rotation slider, the orientation of the grid can be changed. It is recommended to use the same orientation as the crop-row or the main driving direction.
• Horizontal offset: The grid can be shifted horizontally if needed. The mouse wheel can be used to fine-tune each value in smaller steps.
• Vertical offset: The grid can be shifted vertically if needed. The mouse wheel can be used to fine-tune each value in smaller steps.

Once finished with setting the appropriate values for the grid generation, click Start Magic. It usually takes just a few seconds while the AI learns about the structure of the data.

Grid creation menu.

[3] Labeling

In this step, Wanted and Not wanted cells are labeled in order for the AI to start working.

• At least three Wanted cells need to be marked. 
• After selecting the Wanted cells, continue with marking the Not wanted cells, as this is a necessary step. They can be selected via the sidebar or by holding the Shift key to invert the mode temporarily.
• With three labels for each type of cell (Wanted, Not wanted) the Magic Tool will start tracking down similar areas on the whole field instantly.
• More labels can be introduced to train the Magic Tool further and improve its detection performance.
• Once finished with this step, continue with the Cleanup step.

Selecting the Wanted and Not wanted cells for the Labeling step.

[4] Cleanup

Corrections to the results are made during the Cleanup step:

• Select from the Brush section to Add or Delete cells from the selection and adjust the brush size.
• Hold Crtl key, click and drag the cursor to paint multiple cells in a row.
• Hold Shift + Ctrl to invert the mode.
• The Buffer function allows to add or subtract from the selection systematically.
• There is also the option to click on the revert button, on the top-right side of the menu, to return to the initial selection.

Clean up menu.

[5] Convert

Cells can be merged or stay as individual cells. 

• The Area annotation is the default one as it merges the cells, and the resulting annotation is easier to handle.
• Give a descriptive Annotation name.
• Toggle on the button Learn based on my data, to give the option to the Magic Tool to learn from your data. This data will be anonymized training data for more magic in the future!
• Click Convert.

Convert menu.

Results

The detection result is converted as Annotations on the layer that was used for the Magic Tool.

Final result of the Magic Tool as annotations.

Troubleshooting

• If the Magic Tool is taking too long to process, reprocess the dataset with a reduced GSD or export the existing layer with the 50% scale option selected and reimport it into the project
• If there is no detection, export the layer as RGB, reimport it into the project and restart the Magic Tool with it. 
• R00018: The annotations have an invalid format. This error could be caused by special characters in the user's name (accent mark, middle dash)