How to evaluate an AI model - Revision history

Rudolf@tygron.nl at 12:13, 15 October 2025

2025-10-15T12:13:07Z

← Older revision		Revision as of 12:13, 15 October 2025
Line 1:		Line 1:
	[[AI ~~suite~~\|AI]] can be a strong tool to enhance, reinforce, and analyze a variety of datasets for a variety of cases. However, the training-based approach for creating an [[ONNX\|AI model]] means that evaluating the performance and results of such a model requires explicit validation. For this purpose, after training an [[ONNX\|AI model]] with the appropriate training- and testing datasets, it should be set to analyse a third, separate, "evaluation dataset" to validate its accuracy.		[[AI Suite\|AI]] can be a strong tool to enhance, reinforce, and analyze a variety of datasets for a variety of cases. However, the training-based approach for creating an [[ONNX\|AI model]] means that evaluating the performance and results of such a model requires explicit validation. For this purpose, after training an [[ONNX\|AI model]] with the appropriate training- and testing datasets, it should be set to analyse a third, separate, "evaluation dataset" to validate its accuracy.

	The best method for this evalulation is to compare the geographical results of the [[Inference Overlay]] running the [[ONNX\|model]] with an [[Average Overlay]] or [[Combo Overlay]] displaying the known dataset.		The best method for this evalulation is to compare the geographical results of the [[Inference Overlay]] running the [[ONNX\|model]] with an [[Average Overlay]] or [[Combo Overlay]] displaying the known dataset.

Rudolf@tygron.nl at 12:12, 15 October 2025

2025-10-15T12:12:56Z

@@ Line 1: / Line 1: @@
 {{editor steps
-| Create a validation project with a set of areas that can be used for validation. For example areas that define where trees are located with the attribute FOLIAGE_TYPE.
+| Create a validation project with a set of Areas that can be used for validation. The structure of this dataset should be the same as the training and testing datasets, but should be a fully separate dataset.<br>For example areas that define where trees are located with the attribute FOLIAGE_TYPE.
 | Create an [[Inference Overlay]] and run your model contained in the [[ONNX]] file.
-| Create a [[Combo Overlay]] that detects these validation areas and subtracts the results from the [[Inference Overlay]].
+| {{ howtoblock | Create a second [[Combo Overlay]], with the following configuration:
-| The difference between these validation areas and Inference Overlay determine how good the AI model works.
+   | Name: Evaluation Result | Input A: Evaluation Data | Input B: AI Model Overlay | Formula: IF( AND( EQ(A,0), EQ(B,0) ) , NO_DATA , IF( EQ(A,B) , 1 , 0 ) )
-| You can also use a TQL Queries to express this values in number, for example using SELECT_GRIDAREA_WHERE_GRID_IS_ID / SELECT_LANDPOLYGONS_WHERE_AREA_WITH_ATTRIBUTE_IS_FOLIAGE_TYPE.
+}}
 }}

Maxim@tygron.com: Maxim@tygron.com moved page How to validate an AI model to How to evaluate an AI model without leaving a redirect

2025-10-15T08:47:46Z

Maxim@tygron.com moved page How to validate an AI model to How to evaluate an AI model without leaving a redirect

← Older revision	Revision as of 08:47, 15 October 2025
(No difference)

Maxim@tygron.com: Created page with "{{editor steps | Create a validation project with a set of areas that can be used for validation. For example areas that define where trees are located with the attribute FOLIAGE_TYPE. | Create an Inference Overlay and run your model contained in the ONNX file. | Create a Combo Overlay that detects these validation areas and subtracts the results from the Inference Overlay. | The difference between these validation areas and Inference Overlay determine ho..."

2025-10-15T08:26:51Z

Created page with "{{editor steps | Create a validation project with a set of areas that can be used for validation. For example areas that define where trees are located with the attribute FOLIAGE_TYPE. | Create an Inference Overlay and run your model contained in the ONNX file. | Create a Combo Overlay that detects these validation areas and subtracts the results from the Inference Overlay. | The difference between these validation areas and Inference Overlay determine ho..."

New page

{{editor steps
| Create a validation project with a set of areas that can be used for validation. For example areas that define where trees are located with the attribute FOLIAGE_TYPE.
| Create an [[Inference Overlay]] and run your model contained in the [[ONNX]] file.
| Create a [[Combo Overlay]] that detects these validation areas and subtracts the results from the [[Inference Overlay]].
| The difference between these validation areas and Inference Overlay determine how good the AI model works.
| You can also use a TQL Queries to express this values in number, for example using SELECT_GRIDAREA_WHERE_GRID_IS_ID / SELECT_LANDPOLYGONS_WHERE_AREA_WITH_ATTRIBUTE_IS_FOLIAGE_TYPE.
}}