MATLAB: Not able to get accurate true positive (TP), true negative (TN), false positive (FP) and false negative(FN) values in image watermarking

computer visionforgeryimageimage acquisitionimage processingImage Processing Toolboximage segmentationroc curvesimulationtamperingwatermark

I want to understand the basic idea of TP, TN, FP and FN, with respect to following points:
(1) Actually, I am performing tamper detection in image watermarking. At first, some watermark bits are embedded inside the original image to produce the watermarked image. Then, watermarked image is tampered to produce the tampered image.
(2) Now, the watermark bits are extracted from both the watermarked image and the tempred image.
(3) Then, these bits are converted into two binary images and assgined to 2 variables, such as (1) reference and (2) toTest.
(4) Then I am following the below given code to find the TP, TN, FP and FN. The problem is, for any amount of tampering the results for FP and FN comes as 0 only. AM I missing any logic or trick, plz clarify me.
function [TP,FP,TN,FN] = compareBinaryImages( reference, toTest )
% reference = ground truth binary image
% toTest = binary image to be compared to the reference image
if(ndims(reference)~=2 && ndims(toTest)~=2)
error('Inputs must be two 2-dimensional matrices');
end
TP = nnz(reference==1 & toTest==1); % True positive
FP = nnz(reference==0 & toTest==1); % False positive
TN = nnz(reference==0 & toTest==0); % True negative
FN = nnz(reference==1 & toTest==0); % False negative
end

Best Answer

Of course. It all depends on how you do your watermark extraction on the tampered image. For example if you merely subtracted the images and looked for the presence of non-zeros, then even the slightest amound of tampering would show it tampered with. For example if you merely changed the value of one pixel, your algorithm shows that as a true tamper, which it should. Of course comparing some totally different image would also show a difference and you'd say it was a tamper but it might not be. For example if you compared a picture of a beach with a photo of the Grand Canyon it would show it as being tampered with but it's probably not an alteration of your original - it's just a completely different photo. So you need to make sure that you're really comparing something that is likely to be an alteration of your original photo, like by adding noise, rotating it, cropping it, scaling it, changing brightness or colors (overall or just in regions), etc.
Since your False positives and False negatives are always both zero, then whatever you're doing to detect tamper is working fantastic, as long as you're comparing photos that should sensibly be compared. Ideally you would not want any false positives or false negatives. I'm not sure if a true positive is an indication of an original image or a tampered one, but regardless your algorithm is good enough to not make any mistakes since all your predictions are correct (true).