[Math] Proof that $\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$ are independent

generating-functionsprobabilitysamplingstatistical-inferencestatistics

Let $X_i\sim N(\mu,\sigma^2)$ ; where$[i=1,2,\ldots,n]$

$Z_i\sim N(0,1)$ ; where$[i=1,2,\ldots,n]$

Proof that $\bar Z=\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^{n}(Z_i-\bar Z)^2=\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$
are independent, which implies $\bar X$ and $\sum_{i=1}^n(X_i-\bar X)^2$ are independent.

MY ATTEMPT:

I considered $n=2$, and

$$M_{X_1+X_2}(t_1)=M_{X_1}(t_1)M_{X_2}(t_1)$$

$\ast$I did so for the proof but does the statement $X_i\sim N(\mu,\sigma^2)$ ; where$[i=1,2,\ldots,n]$ imply that $X_i's$ are independent?

Moment Generating Function of $X_1+X_2$ and $X_1-X_2$ are
$$M_{X_1+X_2}(t_1)=e^{2\mu t_1+\sigma^2 t_1^2}$$
$$M_{X_1-X_2}(t_2)=e^{\sigma^2 t_2^2}$$
respectively.

Also,

$$M_{X_1+X_2,X_1-X_2}(t_1,t_2)=e^{2\mu t_1+\sigma^2 t_1^2}e^{\sigma^2 t_2^2}=M_{X_1+X_2}(t_1)M_{X_1-X_2}(t_2)$$

since the joint moment generating function factors into the product of the marginal moment generating functions, so $X_1+X_2$ and $X_1-X_2$ are independent which implies that:

$\bullet$ Since $\bar Z=\frac{(\bar X-\mu)}{\sigma}$ is only a function of $X_1+X_2$ and $\sum_{i=1}^{2}(Z_i-\bar Z)^2=\sum_{i=1}^2\frac{(X_i-\bar X)^2}{\sigma^2}$ is only a function of $X_1-X_2$ ,so

$\bar Z=\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^{n}(Z_i-\bar Z)^2=\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$
are independent.

$\bullet$ Since $\bar X$ is only a function of $X_1+X_2$ and $S^2=\frac{1}{2-1}\sum_{i=1}^2(X_i-\bar X)^2$ is only a function of $X_1-X_2$ ,so

Sample mean,$\bar X$ and sample variance,$S^2$ are independent.

$\diamond\diamond\diamond$ We accept the above independence for any arbitrary $n$.

Is the procedure correct?

Best Answer

The way to do this is to show that $\bar X=(X_1+\cdots+X_n)/n$ is independent of $$ (X_1-\bar X, \ldots, X_n-\bar X). $$ Notice that the orthogonal projection of $(X_1,\ldots,X_n)$ onto the $1$-dimensional space $\{ (x,x,x,\ldots,x) : a \in \mathbb R\}$ is $(\bar X,\ldots, \bar X)$, and the orthogonal projection onto the $(n-1)$-dimensional space $x_1+\cdots+x_n=0$ is $(X_1-\bar X, \ldots, X_n-\bar X)$, and those two spaces are orthogonal to each other. Notice also that the probability distribution of $(X_1,\ldots,X_n)$ is spherically symmetric about the point $(\mu,\mu,\ldots,\mu)$, which is in the $1$-dimensional space mentioned above, and which gets mapped to $(0,0,0,\ldots,0)$ by the second projection.

Another way to look at it is this: The random vector $(X_1,\ldots,X_n)$ is jointly normally distributed, i.e. it is so distributed that every linear combination $a_1X_1+\cdots+a_nX_n$ has a $1$-dimensional normal distribution. That implies that any two linear combinations of $X_1,\ldots,X_n$ that are uncorrelated are independent. So then notice that $$ \operatorname{cov}\left( \bar X, \begin{bmatrix} X_1 - \bar X \\ \vdots \\ X_n - \bar X\end{bmatrix}\right) = \begin{bmatrix} 0 \\ \vdots \\ 0 \end{bmatrix}. $$

Related Solutions

[Math] Are the random variables $X + Y$ and $X – Y$ independent if $X, Y$ are distributed normal

Recall that for an $\mathcal{N}(\mu, \sigma^2)$ random variable, the moment generating function of it is $$M(t) = \exp\left(\mu t + \frac{1}{2}\sigma^2t^2\right). \tag{1}$$

By condition, $X + Y \sim \mathcal{N}(2\mu, 2\sigma^2)$ and $X - Y \sim \mathcal{N}(0, 2\sigma^2)$. Therefore by $(1)$, we have: $$M_{X + Y}(t) = \exp\left(2\mu t + \sigma^2 t^2\right), \; M_{X - Y}(t) = \exp\left(\sigma^2 t^2\right).$$

On the other hand, as a bivariate random vector $(X + Y, X - Y)$, its MGF can be computed by definition as follows: \begin{align} & M_{(X + Y, X - Y)}(t_1, t_2) \\ = & E[\exp(t_1(X + Y) + t_2(X - Y))] \\ = & E\left\{\exp[(t_1 + t_2)X] \times \exp[(t_1 - t_2)Y]\right\} \\ = & E\left\{\exp[(t_1 + t_2)X] \right\}\times E\left\{\exp[(t_1 - t_2)Y]\right\} \quad \text{by independence of $X$ and $Y$.}\\ = & M_X(t_1 + t_2) M_Y(t_1 - t_2) \\ = & \exp\left(\mu(t_1 + t_2) + \frac{1}{2}\sigma^2(t_1 + t_2)^2\right)\exp\left(\mu(t_1 - t_2) + \frac{1}{2}\sigma^2(t_1 - t_2)^2\right) \\ = & \exp\left(2\mu t_1 + \sigma^2 t_1^2\right)\exp\left(\sigma^2 t_2^2\right) \\ = & M_{X + Y}(t_1) M_{X - Y}(t_2). \end{align}

Hence $X + Y$ and $X - Y$ are independent.

Compute the $((1-\alpha )\times 100)^{th}$ percentile of the null distribution in R

So what you doing is basically a Monte Carlo Estimate of a specific quantile $\delta_{\alpha ,n}$ so that, $P(\delta >\delta_{\alpha ,n}|H_0)=\alpha.$

Your method would be like this:

Let $F$ be the distribution of $\delta$ under the null distribution.
Simulate random sample $\delta_1,\delta_2,\delta_3,\cdots,\delta_N$ from $F$; $\delta_i\sim F \ \forall i$
Compute the empirical distribution corresponding to them: $$F_N(x)=\frac{1}{N}\sum_{i=1}^N \mathbb{I}_{\delta_i\le x} \left(\stackrel{N\to\infty}{\longrightarrow}F(x)\right)$$
So your estimated quantile would be: $$F_N^{-1}(\alpha)$$ for a large enough N
This quantile $F_N^{-1}(\alpha)$ can be approximated by the "almost sure left inverse" $$\inf\{x\,|\, F_N(x)\geq \alpha\}$$

Your code is almost okay, you have to finally calculate sort(D)[length(D)*alpha]. Also, take a much larger value of $N$ than just 25.

So, finally, in R, your program would be (taking a larger value of N, not 25):

n=10
mu=100
sigma=10
N=1000
alpha=0.05     # Or your favorite value

D=array(0, dim=N)
for(j in 1:N){
    x=rnorm(n,mu,sigma)
    x=x[order(x)]       ## or use sort(x)

    # S=sd(x)           ## sqrt(sum((x - mean(x))^2) / (n - 1))
    # z = c()
    # for(i in 1:n){
    #     z[i] = dnorm((x[i]-mean(x))/S,0,1)
    # }
    z=dnorm(x, mean(x), sd(x))  # R can do it at once

    # num=0
    # for(i in 1:n){
    #     num = num + abs(z[i]-i/n)
    # }
    num = sum( abs(z - seq(1,n)/n ) ) 

    # den = 0
    # for(i in 1:n){
    #     den = den + max(z[i],i/n)
    # }
    den = sum( pmax(z, seq(1,n)/n) )

    D[j]=num/den
}
sort(D)[N*alpha]

I have commented on some parts of your codes and compacted them.

Best Answer

Related Solutions

[Math] Are the random variables $X + Y$ and $X – Y$ independent if $X, Y$ are distributed normal

Compute the $((1-\alpha )\times 100)^{th}$ percentile of the null distribution in R

Related Question