Expected Supremum of Average – Probability Insights

Question

Is there either a closed form (in terms of the moments of $X_1$, say) or good bounds on
$$
\mathbb{E} \sup_{k \leq n} \frac{1}{k} \sum_{i=1}^k X_i,
$$
where $X_i$ are iid and arbitrarily nice? (In my specific application, $X_i$ are given by $(B_i – p)^2$, where $B_i$ are iid Bernoulli variables with mean $p$.) I am particularly interested in the correct functional dependence on $n$; e.g., is there a constant bound that holds for all $n$?

Answer 1

You're asking about maximal inequalities. These are known in more generality for measure-preserving transformations. As has already been pointed out, in your special case, you can expect to get a constant bound. The averages very quickly approach the limit, so you're looking at the average of the max of the first few terms together with the limiting value.

In general, for measure-preserving transformations, if the $X_k$ are just $L^1$ (even in the iid case), the expectation of the supremum can diverge logarithmically. If the $X_k$ have some higher moments: $L^p$ for any $p>1$, then you get the bound $\|M((X_k))\|_p \le C_p \|X_1\|_p$, where $C_p$ is some constant with a $p-1$ in the denominator.