If $p<1$ and $X$ is a random variable distributed according to the geometric distribution $P(X = k) = p (1-p)^{k-1}$ for all $k\in \mathbb{N}$, then it is easy to show that $E(X) = \frac 1p$, $\mathop{Var}(X)=\frac{1-p}{p^2}$ and $E(X^2) = \frac{2-p}{p^2}$.
Now consider a "conditional" geometric distribution, defined as follows (if there is standard terminology for this, let me know and I'll call it that):
- Fix a set $J\subset \mathbb{N}$ and a number $\mu>0$ (this will eventually be large).
- Let $P(X=k) = C \gamma^k$ if $k\in J$ and $P(X=k)=0$ otherwise, where $C>0$ and $\gamma<1$ are chosen so that probabilities sum to $1$ and $E(X) = \mu$.
I'm trying to understand how $E(X^2)$ (or equivalently, $\mathop{Var}(X)$) depends on $J$ and $\mu$. In the case where $J=\mathbb{N}$ the standard results show that $p=\frac 1\mu$ and so $E(X^2) = \mu^2(2-\frac 1\mu)$. I'm interested in the case where $\mu$ becomes very large and would like to obtain a similar estimate $E(X^2) \approx A\mu^2$, for some constant $A>1$, in a more general setting.
The example I'm working with at the moment is $J= \{2^n \mid n\in \mathbb{N}\}$, but ideally I'd like some conditions on the set $J$ that would guarantee an estimate of the above form.
Is there a standard name for these distributions, or a reference where I can read more about them? Are estimates of this form known?
Edit: As Brendan McKay pointed out below, this boils down to understanding the behaviour of the function $g(\gamma) = \sum_{j\in J} \gamma^j$, and in fact the issue that motivated the question I posed can be stated more directly in terms of this function.
The condition $E(X) = \mu$ is equivalent to the equation $\mu = \gamma g'(\gamma) / g(\gamma)$, which determines $\gamma$ implicitly as a function of $\mu$. We would like to understand how $g(\gamma)$ grows as $\mu\to\infty$, and hence $\gamma\to 1$. (In particular, this means we're really interested in the case where $J$ is infinite.)
In the case $J=\mathbb{N}$, one has $g(\gamma) = \frac\gamma{1-\gamma} = 1 – \frac 1{1-\gamma}$, and so $\mu = \gamma (\frac{\gamma}{(1-\gamma)^2}) (\frac{1-\gamma}\gamma) = \frac{\gamma}{1-\gamma}$, so that in fact $g(\gamma(\mu)) = \mu$ and the two quantities go to infinity together.
In the more general case a reasonably simple argument shows that $\lim_{\mu\to\infty} g(\gamma(\mu)) = \infty$ provided $J$ is infinite, but it's not at all clear to me how the rate at which $g$ grows (in terms of $\mu$) depends on $J$ for more general sets.
That's the original motivation — after some messing around we decided that we could figure out the growth rate if we knew something about $E(X^2)$ as suggested above, and since it was phrased in terms of what seemed to be a reasonably natural probability distribution, we decided to ask it in that form. But now you have the whole story…
Best Answer
I'm not sure what you really want but here is a couple of simple minded inequalities that can serve as a baseline.
Below $g=\sum_{k\in J}\gamma^k$, $M=\sum_{k\in J}k\gamma^k$, so $\mu=\frac Mg$. We'll need the counting function $F(n)=\#\{k\in G: k\le n\}$ of the set $J$. I will assume that $F$ is extended as a continuous increasing function to the set $[1,+\infty)$ and that $g\ge 1$.
1) For every $N$, we have the trivial estimate $g\le F(N)+\frac MN$. Taking $N=2\mu$, we get $g\le F(2\mu)+\frac g2$, i.e.,
$$ g\le 2F(2\mu) $$
2) Let $\nu$ satisfy $F(\nu)=3g$. Since $g\ge F(\nu)\gamma^\nu$, we conclude that $\gamma^\nu\le \frac 13$ so $1-\gamma>\frac 1\nu$. Now, for every $N$, we have $$ M\le Ng+(N+\frac 1{1-\gamma})\frac 1{1-\gamma}\gamma^N\le Ng+(N+\nu)\nu e^{-N/\nu}\. $$ Since we clearly have $\nu\ge F(\nu)=3g$, we can choose $N=\nu\log\frac\nu g\ge \nu$. For this choice, the second term on the right is at most $2Ng$, so, dividing by $g$ we get $\mu\le 3N$, i.e., $$ \mu\le 3F^{-1}(3g)\log\frac{F^{-1}(3g)}{g} $$
Examples of what these inequalities yield:
1) Dense set ($F(n)\approx n$). Then $g\approx\mu$
2) Power lacunarity ($F(n)\approx n^p$, $0<p<1$). Then $g$ is between $\mu^p(\log\mu)^{-p}$ and $\mu^p$ up to a constant factor.
3) Geometric lacunarity ($F(n)\approx\log n$). Then $g\approx \log\mu$.
As you see, one can lose a logarithm sometimes but the advantage is that I do not make any regularity assumptions here. Of course, if $F$ is regular enough, you can, probably, do a bit better.