This article introduces the Magic Tool in PIX4Dfields, an Artificial Intelligence (AI)- assisted selection tool for quickly detecting and selecting weed nests, damage, and other anomalies in orthomosaics and index layers.
IN THIS ARTICLE
How to access the Magic tool
[1] Layer selection
[2] Grid creation
[3] Labeling
[4] Cleanup
[5] Convert
Results
Troubleshooting
How to access the Magic Tool
Access the tool by clicking the Magic icon located next to the
Index icon.
[1] Layer selection
- Hide or show annotations that may help when using this tool.
- Select the Source layer where Magic Tool will run on top.
- Select a Boundary of the area from the Source layer where the Magic Tool will work.
- Select Obstacles if needed. These zones will be discarded by the Magic Tool.
[2] Grid creation
A grid needs to be created for classification purposes when using the Magic Tool. The definition of the grid is based on the following aspects:
- Cell size: Select a cell size that roughly represents the structures that need to be detected on the map. When planning a herbicide spot spraying mission, the cell size could match the implements section or working width.
- Rotation: With the rotation slider, the orientation of the grid can be changed. It is recommended to use the same orientation as the crop-row or the main driving direction.
- Horizontal offset: The grid can be shifted horizontally if needed. The mouse wheel can be used to fine-tune each value in smaller steps.
- Vertical offset: The grid can be shifted vertically if needed. The mouse wheel can be used to fine-tune each value in smaller steps.
Once finished with setting the appropriate values for the grid generation, click Start Magic. It usually takes just a few seconds while the AI learns about the structure of the data.
[3] Labeling
In this step, Wanted and Not wanted cells are labeled in order for the AI to start working.
- At least three Wanted cells need to be marked.
- After selecting the Wanted cells, continue with marking the Not wanted cells, as this is a necessary step. They can be selected via the sidebar or by holding the Shift key to invert the mode temporarily.
- With three labels for each type of cell (Wanted, Not wanted) the Magic Tool will start tracking down similar areas on the whole field instantly.
- More labels can be introduced to train the Magic Tool further and improve its detection performance.
- Once finished with this step, continue with the Cleanup step.
[4] Cleanup
Corrections to the results are made during the Cleanup step:
- Select from the Brush section to Add or Delete cells from the selection and adjust the brush size.
- Hold Crtl key, click and drag the cursor to paint multiple cells in a row.
- Hold Shift + Ctrl to invert the mode.
- The Buffer function allows to add or subtract from the selection systematically.
- There is also the option to click on the revert button, on the top-right side of the menu, to return to the initial selection.
[5] Convert
The detections can be converted to:
- Area annotation: The detections will be saved as area annotations. This is useful when detecting weed nests, damage in crops, measuring areas, etc.
- Boundary: the detections will be saved as a boundary. This is useful when trying to mark the field boundary by using the Magic tool.
- Obstacle: the detections will be saved as obstacles. This is useful when using the Magic Tool to detect trees that should be avoided in the mission, for example.
- It is recommended that One sub polygon per grid cell be toggled off. Cells will be merged, resulting in a simpler geometry.
- Give a descriptive Annotation name.
- Toggle on Learn based on my data to give the Magic Tool the option to learn from your data. This data will be anonymized training data for more magic in the future!
- Click Convert.
What to do with Magic Tool outputs?
Magic tool outputs can be exported from the Export menu.
When doing spot spraying or variable-rate applications, it is recommended to create a Targeted Operation, selecting the Annotations created with the Magic Tool as input.
- Go to Operations.
- Select the second option, "Use existing annotations to pre-fill the grid".
- Magic tool output as Boundaries and obstacles can also be used in Targeted Operations
Once Magic Tool outputs have been used in Targeted Operations, the files will be ready for use in tractors, ground sprayers, spraying drones, and other equipment.
Troubleshooting
- If the Magic Tool is taking too long to process, reprocess the dataset with a reduced GSD or export the existing layer with the 50% scale option selected and reimport it into the project
- If there is no detection, export the layer as RGB, reimport it into the project and restart the Magic Tool with it.
- R00018: The annotations have an invalid format. This error could be caused by special characters in the user's name (accent mark, middle dash)