I tried to run Land change modeler with an MLP neural network option.
I received the accuracy value of 87 %. I just wonder how is this value calculated?
Is it possible (how) to calculate the ROC statistics (AUC) and for example hits and false alarm rates at this point for this model?
For an explanation of the MLP accuracy rate, please refer to the MLP help menu. You may also find that page 270 of the IDRISI Selva Tutorial (Page 271 of the IDRISI Taiga tutorial) also provides some clarity on this matter.
To calculate the ROC statistic you can use the ROC module found under GIS Analysis > Change/Time Series > ROC.
As for viewing the hits (agreement between predicted change and true change), misses (no change predicted but change actually occurred), and false alarms (change predicted but no change actually occurred), you can use the Validation panel in the Change Prediction tab of LCM. There is additional information on the Validation panel in Exercise 6-4 of the IDRISI Tutorial. You can also use the CROSSTAB module.
David
Clark Labs Tech Support
Thanks for the answer David.
Howerer all options that you mentioned are valid only for the Change predictions.
But how it is possible to validate just the Change potential? The only information is the MLP accuracy rate. How is this rate calculated?
Kind regards,
Rob.
After having created your Transition Potentials, create a mask (i.e., a Boolean image) for net persistence and a mask for net change. You will run HISTO twice, both times using your Transition Potential image as your input. Include the net persistence and net change images as a masks (once each). By selecting a numerical output in the HISTO module, you will be able to graph the net persistence against the net change. Your graph will show frequency as the y-axis and probability as the x-axis. If your graph shows a high peak at both extremes of the x-axis, then it shows that you have more meaningful driver variables. Frequent peaks in the middle of the x-axis show less meaningful driver variables.
In the Bibliography section of our website, we have a referenced a paper that further explains how to validate change potential, authored by Sangermano, Toledano, and Eastman (Clark Labs Home > Resources > Bibliography > By Subject > Land Change Modeler). The link for this paper can be found on this page: http://www.clarklabs.org/resources/Bibliography-by-Subject.cfm#Land%20Change%20Modeler
For further information on how the MLP accuracy rate is calculated, there is an extensive explanation in the Notes section of the MLP entry in the Help System (Help > Contents > MLP).
Laura
Clark Labs Tech Support
Thanks for your answer, was great. But the problem is that the Transition Potential map as a output of the MLP has masked out locations that recorded change, so is not possible to produce histogram for net change. Isnt it so?
That is correct, you need a third date to use the aproach above. So you will be extracting the change between T2 and T3 and use that for the histogram.
Dear Clark Labs Tech Support Team
I also confused how Accuracy Rate has been calculated. From the IDRISI Tutorial does not tell how it was formulated. What i only can see is from help in the model mentioned about E(A) = 1/T+P , where E(A) =Expected accuracy, T = number of transition in the sub-model, and P = number of persistence class = the number o f"from" classes in the sub-model. I really don't understand what are T and P in terms of units and numbers. Could you please have further details about this or provide us a simple calculation for a class of land use? I have been looking for this formula but can't fingered out any publications. Similarly with Skill measure, I am not sure but the formula is look alike Kappa statistic equation. Could you open up my mind about these above question? Many thanks for your helps.
Dear all
I am using new version Terrset 2020. When run transition sub model using MPL model gives me very lower accuracy 44%. However other model such as SVm and Decisuin forest giver more than 80%. why this?