Practice talk for ICML 2012.

Abstract: Precision-recall (PR) curves and the areas under them are widely used to summarize machine learning results, especially for data sets exhibiting class skew. They are often used analogously to ROC curves and the area under ROC curves. It is already known that PR curves vary as class skew varies. What was not recognized before this paper is that there is a region of PR space that is completely unachievable, and the size of this region varies only with the skew. This paper precisely characterizes the size of that region and discusses its implications for empirical evaluation methodology in machine learning.