I have tried different ways to find a way to deal with the following problem. I also tried to find out what happen when the determinat is zero by calculating it but I didn't get anything. Any ideas are appreciated for first steps.
Thanks in advance.
Reference: The Arithmetic of Elliptic Curves, By J. H. Silverman (Ex. 3.9 page 106)
Let $E/K$ be a curve over a field $K$ with $char(K) \neq 2,3$, and fix a homogeneous Weierstrass equation for $E$,
$$F(X_0,X_1,X_2) = X_1^2X_2-X_0^3-AX_0X_2^2-BX_2^3=0,$$ i.e., $x=X_0/X_2$ and $y=X_1/X_2$ are affine weierstrass coordinates. Let $P \in E$. Prove that $[3]P=0$ if and only if the Hessian matrix $$\left( {\partial^2F \over \partial
X_i X_j} (P)\right)_{0\le i,j \le 2}$$ has determinat $0$.
Best Answer
Short answer: the point at infinity is an easy case. For any affine point, notice that $\det H=8\psi_3$, where $\psi_3$ is a division polynomial. From standard properties of division polynomials, the result follows.
Longer answer: in case you don't immediately see the connection with the division polynomial, you could take the following approach. This is what I started out with, but essentially what we're doing is deriving $\psi_3$.
Let's first assume that $P\neq \mathcal{O}$ and $[2]P\neq\mathcal{O}$, in those cases we would want to show that the Hessian always respectively never has determinant zero. This is done at the end. Write $P=(x,y)$.
In this case we know that the tangent line contains a third point, and it is not $\mathcal{O}$. In fact, we have exact formulas to compute the third point (see page 54 of Silverman, specialized to $\operatorname{char}\neq2,3$), namely: $$\begin{align*} \lambda &= \frac{3x^2+A}{2y}, \\ x_{2} &= \lambda^2-2x, \\ y_{2} &= \lambda (x-x_2)-y. \end{align*}$$ Note that $(x_2,y_2)$ is generally derived by taking it as the third intersection point of the tangent line at $P$. Now we know that $[3]P=\mathcal{O}\iff x_2=x$ and $y_2=-y$. Substituting the curve equation into $x=\lambda^2-2x$, we obtain $$3x^4+6Ax^2+12Bx-A^2=0.\quad (\text{this is }\psi_3\text{!})$$ Now I will not post the whole computation here (I will if you really want me to!), but as it turns out the left-hand side is almost the determinant of the Hessian. So we can conclude that if $[3]P=\mathcal{O}$, then the determinant of the Hessian vanishes.
On the other hand, if the determinant vanishes, we can revert all operations above to show that $x=\lambda^2-2x$. Hence $x_2=x$. From this it immediately follows that $y_2=-y$, hence $[2]P=-P$, and $[3]P=\mathcal{O}$.
Finally a note on $P=\mathcal{O}$ and $[2]P=\mathcal{O}$. In the first case we write $P=(0:1:0)$ and it is immediate from a computation that the Hessian has zero determinant. If $[2]P=\mathcal{O}$, then we know that $y=0$. We can compute the determinant as $D(x)=24Ax^2+72Bx-8A^2$. As $y=0$, also $F(x)=x^3+Ax+B=0$. It turns out that $$\operatorname{Res}(D,F)=-\Delta^2,$$ where $\Delta$ is the (non-zero!) discriminant of the curve. Hence $D$ and $F$ do not share a root, and we conclude that $D(x)\neq 0$.