machine learning - How to classify true negative from a video?

Question

For a performance measuring purpose I am trying to draw ROC curve. In ROC curve I have to plot False Positive Rate (FPR) in x-axis and True Positive Rate (TPR) in y-axis. As we know,

FPR = FP/(FP+TN)

So in the following picture how can i detect True Negative(TN) ? I have used HOG classifier to detect human. I marked with rectangle 1,2,3,4,5,6(or should be 7) to show the human objects that should be ignored and not to classify as human. and I think those are True Negative.

In this picture i want to say my assumption,as we know,

False negative: Result should have been positive, but is negative.

False positive: Result should have been negative, but is positive.

True positive: Result should have been positive and is positive.

True negative: Result should have been negative and is negative

So i think in this frame FP = 0, TP = 0, FN = 0 but not sure about TN, is it 6 or 7 or anything other? Please correct me also about FP, TP, and FN if i am wrong. I saw this question How to categorize True Negatives in sliding window object detection? which was really helpful but still i have to calculate FPR for this scenario.

Answer 1

You cannot calculate these values from such image, you need more data (knowledge what is actually happening). But what you need is probably just total amount of these windows, which is some constant N. Now, it seems like all these windows are wrong (none is on the human), thus:

• FP = 6 (your method claims there are 6 people, but none of these claims is valid since they are completely off - however if this is just visualization issue, and method actually captured valid people, this 6 should be moved to TP instead)
• TP = 0 (it does not correctly mark any human)
• FN = 10 (if I counted correctly there are 10 people at this image, and all these are missing)
• TN = N - 16, where N is number of all analized windows, since all of them are correctly classified as "lack of human" up to 10 FNs and 6 FPs, which add up to these 16.

In general

• FP = how many actual not humans are marked "human"
• TP = how many actual humans are marked "human"
• FN = how many actual humans are correctly ignored (not marked "human")
• TN = how many actual not humans are correctly ignored (not marked "human")