I encountered the following proposition about the variance of the sample mean in my text book.
$$\text{Var}(\bar{X}) = \frac{\text{Var}(X_1)}{n} = O(n^{-1}).$$
From this I understand big $O$ notation in statistics to mean the same as it does in $\text{CS}$ (namely $\exists C$ such that $\frac{\text{Var}(X_1)}{n} \leq Cn^{-1}$ for $n \to \infty$) which makes sense because $\text{Var}(X_1)$ is a constant.
However, later on I encounter the following.
$$\text{Var}(S^2) = \mathbb{E}[(S^2-\sigma^2)^2] = \frac{1}{n}(\mathbb{E}[(X_1-\mu)^4]-\frac{n-3}{n-1}\sigma^4)= O(n^{-1}).$$
Which doesn't seem to make sense since ($\mathbb{E}[(X_1-\mu)^4]-\frac{n-3}{n-1}\sigma^4)$ obviously varies with $n.$ Am I missing something?
Best Answer
I think you misunderstand what the big-O notation means. For two functions $f,g$ over naturals we say that $f\in O(g)$ if there exists a constant $C$ such that $f(n)\leq Cg(n)$ eventually.
So if $f$ is bounded by say $M$, then by the definition $f(n)\leq M$ for any $n$ and thus $f\in O(1)$. In particular $\sin(n)$ is $O(1)$ even though it varies with $n$.
And so your $f(n)=\mathbb{E}[(X_1-\mu)^4]-\frac{n-3}{n-1}\sigma^4$ component is bounded, thus it is $O(1)$. Therefore $\frac{1}{n}f(n)$ is $O(\frac{1}{n})$ since the big-O preserves multiplication by nonnegative functions.