I have an object detection algorithm for which I would like to plot an ROC curve. For this, I need the values of the fall-out corresponding to values of recall.
The false positive rate, or fall-out, is defined as
$$\text{Fall-out}=\frac{FP}{FP+TN}$$
In my data, a given image may have many objects. So, almost every image has at least one box. I am counting a predicted box as a true positive if its IOU with a truth box is above a certain threshold, and as a false positive otherwise. Any truth box with no prediction box having an IOU above a threshold counts as a false negative.
However, for the denominator, how is a "true negative" defined here? In the context of object detection, what does it mean to say that something is a true negative? Can any bounding box not detected, that does not correspond to an actual object (with an IOU above a certain threshold), be called a true negative? If so, wouldn't the number of true negatives be infinite?
From what I understand, $TN$ would be the number of "non-objects" that were not detected. How can this be quantified? Is it defined in some particular way, or undefined, or infinity? Would it even be possible to plot an ROC for this?
How can I do this?
Thank you.
Best Answer
There are two errors that your algorithm can make. The first error is detecting an object when it's not there. This is the false positive. The second error is not detecting an object when it's there. This is the false negative.
To compute the false positive rate you want to compute how often it detected an object when the object was not there. This is the quotient of the false positives and all potential false positives: i.e.
$\frac{\text{detect box when no box}}{\text{all no box}}$
To compute the false negative rate you compute how often an object is not detected divided by how often the object could have been detected.
$\frac{\text{detect no box when box}}{\text{all box}}$
The key distinction between the false positive and false negative rate is which situation / hypothesis is true. False positives exist only when there are no boxes. False negatives exist only when there are boxes.