maximum likelihoodparameter estimationstatistics
Just would like to know whether the following statement is true or not:
For a discrete random variable, the numerator of the likelihood ratio test (which is a supremum on a set) is equal to the maximum likelihood of the sample for the null hypothesis.
Please provide a counter example if false. I, unsuccessfully, tried proving that the supremum is the maximum by proving that the parameter set will be compact for a discrete random variable.
Thanks for the help!
You're given the test-statistic and the decision rule for the test, which is reject $H_0$ if $y_{max} > k$. Now you want to find $k$ such that the test has size $\alpha$; that is, you want to find the minimum $k$ such that the probability of rejecting the null if it is true is less than $\alpha$.
In math, you want to find the smallest $k$ such that $$ P(y_{max} > k \, | \, \theta = \theta_0) \le \alpha. $$ The probability above is $$ P(y_{max} > k \, | \, \theta = \theta_0) = 1 - P(y_{max} \le k \, | \, \theta = \theta_0) = 1 - P(Y_1 \le k)^n = 1 - \left( \frac{k}{\theta_0} \right)^n. $$ Notice that the second equality follows because the maximum of a random sample is less than $k$ if and only if all of the $Y_i$ are less than $k$, and since they're all independent this is just $P(Y_1 \le k)^n$.
Alright, so now you can solve for $k$ and get $$ k \ge \theta_0 (1-\alpha)^{1/n}. $$ This inequality gives all the possible values of $k$ such that the test has size $\alpha$. You actually wanted the $k$ that maximizes the power. That $k$ should be $\theta_0 (1-\alpha)^{1/n}$ (since it is the smallest $k$ such that the test has size $\alpha$).
You can verify that The LR test will reject the null hypothesis $H_0: p \ge 0.0408$ in favor of the alternative $H_a: p < 0.0408$ for sufficiently small values of $\hat p = x/n,$ which is to say small values of $x.$
Then for $n = 1225, \hat p=0.020408,$ the P-value of your LR test will be $P(X \le 25\, |\, n=1225, p=0.0408) \approx 0.$ This P-value is much smaller than $0.05 = 5\%,$ so you reject $H_0$ in favor of $H_a.$
R code for figure:
n = 1225; p=0.0408; x = 0:90; PDF = dbinom(x,n,p)
hdr = "PDF of BINOM(1225, 0.0408)"
plot(x, PDF, type="h", col="blue", lwd=2, main=hdr)
abline(h=0, col="green2")
abline(v=0, col="green2")
abline(v = 25.5, col="red", lwd=2, lty="dotted")
Because $n$ is so large, you could approximate this P-value with a normal approximation. The exact binomial probability can be found from R as shown below:
pbinom(25, 1225, 0.0408)
[1] 5.508296e-05
Note: Testing with a discrete probability distribution such as binomial, it not not usually possible to do a (nonrandomized) test at exactly the 5% level.
But if you used $c = 38$ as the 'critical value' for the test (that is, rejecting $H_0$ for $X\le c),$ you would have significance level $\alpha=4.43\%,$
qbinom(.05, 1225, .0408)
[1] 39
pbinom(39, 1225, .0408)
[1] 0.06102795
pbinom(38, 1225, .0408)
[1] 0.04434609
If you use a normal approximation to approximate the critical value, it may seem that you are testing at level 5%, but there are no possible (integer) values of $x$ that give z-scores very near to -1.645.
qnorm(.05)
[1] -1.644854
Best Answer
The answer to your question is no. To see why, let's look at a simple example. Suppose the proportion of people with red hair among a population is $\theta\in [0,1]\cap \mathbb{Q}$. Let $\Theta_0=\Big\{\frac{1}{n}\Big\}_{n\in\mathbb{N}}.$ We perform the hypothesis test $H_0:\theta \in \Theta_0$ versus $H_a:\theta \notin \Theta_0$ by taking a SRS of size $n=5$ from our population. Suppose that, among our sample of five individuals, we observe nobody with red hair. Our likelihood function becomes $\mathcal{L}(\theta)=(1-\theta)^5$. Notice $\max_{\theta \in \Theta_0}\mathcal{L}(\theta)$ does not exist whereas $\sup_{\theta \in \Theta_0}\mathcal{L}(\theta)=1$