arXiv (Cornell University)· 2020· Preprint
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
- 4,431citations
- 2020year
Short summary
Commonly used evaluation measures like Precision, Recall, and F-Measure are biased and can misrepresent system performance, potentially showing worse systems as better.
AI-generated from the title and abstract; the full text is not read.
Key points
- Standard metrics (Precision, Recall, F-Measure) are biased and can obscure true system performance.
- A system performing objectively worse can appear better using these common measures.
- Informedness and Markedness are proposed as more reliable measures of prediction accuracy.
- Connections between Informedness, Markedness, Correlation, and Significance are demonstrated.
AI-generated from the title and abstract; the full text is not read.
Abstract
Commonly used evaluation measures including Recall, Precision, F-Measure and Rand Accuracy are biased and should not be used without clear understanding of the biases, and corresponding identification of chance or base case levels of the statistic. Using these measures a system that performs worse in the objective sense of Informedness, can appear to perform better under any of these commonly used measures. We discuss several concepts and measures that reflect the probability that prediction is informed versus chance. Informedness and introduce Markedness as a dual measure for the probability that prediction is marked versus chance. Finally we demonstrate elegant connections between the concepts of Informedness, Markedness, Correlation and Significance as well as their intuitive relationships with Recall and Precision, and outline the extension from the dichotomous case to the general multi-class case.
The authors' abstract, as published at the source. arXiv (Cornell University), 2020 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Computational Theory and Mathematics
Computational Theory and MathematicsComputer Science