Renjith - the problem is with the r
r = [round(abs((i^2)-(j^2)+(a*i)+(b*j))),round(abs((2*i*j)+(c*i)+(d*j)))]
There is no guarantee that the either component of this array will be valid indices into the image
ggg(i,j)=in_img(r(1)+1,r(2)+1);
What you can do is to find the modulus m and n for the first and second component of r respectively.
r = mod([round(abs((i^2)-(j^2)+(a*i)+(b*j))),round(abs((2*i*j)+(c*i)+(d*j)))], [m,n]);
The same would need to be done for the Henon Mapping
r = mod([round(abs(1-(a*(i^2))+j)),round(abs(b*i))],[m n]);
Though I don't know how this will impact the encryption...
Try the above and see what happens!
Best Answer