I don't think details are required since
A=magic(n)-ones(n)*((1+n*n)/2)
is changed into an antisymmetric matrix, any such A matrix must satisfy (basic math.. etc)
det(A) = -1^n * det(A)
since n is odd, det(A) must be zero (thus, A is singular). Changing A from magic(n) to (magic(n)-ones(n)*((1+n*n)/2) ) as mentioned in the question is enough to destroy the symmetry of A.
Yet, since this is too basic, and it works the same for magic(n) with n is odd or even, (also, produces antisymmetric), I'm afraid you already know this. I tried (quickly, to be honest) other means like the nice arguments above, but didn't got anything useful so I thought to share, it might help. Regards.
Best Answer