The Fisher Information is defined as
$${\left(\mathcal{I} \left(\theta \right) \right)}_{i, j}
=
\operatorname{E}
\left[\left.
\left(\frac{\partial}{\partial\theta_i} \log f(X;\theta)\right)
\left(\frac{\partial}{\partial\theta_j} \log f(X;\theta)\right)
\right|\theta\right]$$
(the question in the post you linked to states mistakenly otherwise, and the answer politely corrects it).
Under the following regularity conditions:
1) The support of the random variable involved does not depend on the unknown parameter vector
2) The derivatives of the loglikelihood w.r.t the parameters exist up to 3d order
3) The expected value of the squared 1st derivative is finite
and under the assumption that
the specification is correct (i.e. the specified distribution family includes the actual distribution that the random variable follows)
then the Fisher Information equals the (negative of the) inverted Hessian of the loglikelihood for one observation. This equality is called the "Information Matrix Equality" for obvious reasons.
While the three regularity conditions are relatively "mild" (or at least can be checked), the assumption of correct specification is at the heart of the issues of statistical inference, especially with observational data. It simply is too strong a condition to be accepted easily. And this is the reason why it is a major issue to prove that the log-likelihood is concave in the parameters (which leads in many cases to consistency and asymptotic normality irrespective of whether the specification is correct -the quasi-MLE case), and not just assume it by assuming that the Information Matrix Equality holds.
So you were absolutely right in thinking "too good to be true".
On the side, you neglected the presence of the minus sign -so the Hessian of the log-likelihood (for one observation) would be negative-semidefinite, as it should since we seek to maximize it, not minimize it.
To expand on my comment, you could have worked directly with the score for the combined sample $\mathbf X$.
From the joint pdf of $\mathbf X$, it follows that the score function is $$\frac{\partial}{\partial\theta}\ln f(\mathbf X\mid\theta)=\sum_{i=1}^n \ln X_i+\frac{n}{\theta}$$
So the asymptotic distribution of the score is equivalent to the limiting distribution of $\sum\limits_{i=1}^n \ln X_i$.
By CLT, $$\frac{\sum_{i=1}^n \ln X_i+\frac{n}{\theta}}{\sqrt{\frac{n}{\theta^2}}}\stackrel{L}\longrightarrow N(0,1)$$
Note that the Fisher information in $\mathbf X$ is actually $$I(\theta)=-n E_{\theta}\left[\frac{\partial^2}{\partial\theta^2}\ln f(X_1\mid\theta)\right]=\frac{n}{\theta^2}$$
So the pivot you are looking for is eventually $$\frac{1}{\sqrt n}\left(\theta\sum_{i=1}^n \ln X_i+n\right)\stackrel{L}\longrightarrow N(0,1)$$
Exact confidence intervals for $\theta$ are also availabe, a suitable pivot being
$$-2\theta\sum_{i=1}^n \ln X_i\sim \chi^2_{2n}$$
Best Answer
Because your random variable is still $X$ with pdf $p(X|\beta)$.
When you take the expectation with respect to $X$ of $\frac{ \partial }{ \partial \beta } \ln \mathcal{L}(X|\beta)$ this is just a function of $X$, hence the pdf is $p(X|\beta)$.
I see it's also called "law of the unconscious statistician".