Solved – Tail bounds for Beta distribution

beta distributionprobability-inequalities

Say $X\sim\mathrm{Beta}(\alpha,\beta)$. Are there any "nice" closed form upper bounds for the tail probability $P(X\geq\epsilon)$, that are reasonably tight when $\beta$ is large? By "nice" I mean involving only elementary functions, and not for instance the incomplete beta function.

In my specific setting I have $\alpha=1/2$ and $\beta=d/2$ for some (typically large) integer $d$, and so one way of bounding the tail is to write $X=\frac{Y}{Y+Z}$ where $Y\sim\chi^2_1$ and $Z\sim\chi^2_d$, and using an upper bound on $Y$ and lower bound on $Z$ (e.g. from here: What are the sharpest known tail bounds for $\chi_k^2$ distributed variables?). This gives an answer, but it's somewhat messy (requiring the small $d$ case to be handled separately, etc). Is there a cleaner way?

Thanks.

Best Answer

We write $I_{\epsilon}(A,B,\epsilon)$ the incomplete beta function. Thus $P(X>\epsilon) = 1-I_{\epsilon}(A,B,\epsilon)$

A simple bound is

$$\frac{2\sqrt{\epsilon}}{2\sqrt{\epsilon}+B\left(\frac{1}{2},\frac{d}{2}\right)}< I_\epsilon\left(\frac{1}{2},\frac{d}{2},\epsilon\right)<\frac{2 \sqrt{\epsilon}}{B\left(\frac{1}{2},\frac{d}{2}\right)}$$

A tighter lower bound (so upper bound for the tail) would be

$$\frac{2 \left(\sqrt{\epsilon} B\left(\frac{1}{2},\frac{d}{2}\right)+2 \epsilon\right)}{2 \sqrt{\epsilon} B\left(\frac{1}{2},\frac{d}{2}\right)+B\left(\frac{1}{2},\frac{d}{2}\right)^2+4 \epsilon}$$

Mathematica will spit them out for you with

1 - 1 / Normal[Series[1/(1 - BetaRegularized[\epsilon, 1/2, d/2]), {\epsilon, 0, n}]]

Related Solutions

Solved – Order statistic for beta distribution

Let the parent random variable $X \sim Beta(a,b)$ with pdf $f(x)$:

_{(source: tri.org.au)}

Then, given a sample of size $n$, the pdf of the $1^{st}$ order statistic (sample minimum) is:

_{(source: tri.org.au)}

... and the pdf of the $n^{th}$ order statistic (sample maximum) is:

_{(source: tri.org.au)}

where:

I am using the OrderStat function from the mathStatica package for Mathematica to automate the nitty-gritties
Beta[x,a,b] denotes the incomplete Beta function $B_x(a,b) = \int _0^x t^{a-1} (1-t)^{b-1} d t$
and the domain of support is (0,1).

Solved – Credible set for beta distribution

A level $\alpha$ "Highest Posterior Density" (HPD) interval for a (posterior) distribution $F$ with (continuous) density $f$ and mode $\mu$ is an interval $I=[x,y]$ containing $\mu$ for which

$1-\alpha$ of the probability is in the interval: $F(I) = F(y) - F(x) = 1-\alpha$.
The densities are the same at either end: $f(x) = f(y)$.

Among the various strategies to find $I$, one that stands out as generally effective is the following.

Choose a reasonable starting value for $x$, such as $F^{-1}(\alpha/2)$.
Define the "$\alpha$ offset" of $x$ to be the point $y$ for which $[x,y]$ has probability $1-\alpha$. Thus

$$y = F^{-1}(F(x) + 1 - \alpha)$$

provided $F(x) \le \alpha$.
Search for $x$ in the interval $(-\infty, F^{-1}(\alpha))$ at which $f(x) = f(y)$. The unimodality of $F$ and the continuity of $f$ guarantee such an $x$ exists and is unique.

The search (3) can be carried out in practice by minimizing $(f(y)-f(x))^2$ plus a penalty term in case the probability of $[x,y]$ is not exactly $1-\alpha$. (The penalty term is useful in case the search procedure provides a candidate value of $x$ for which $F(x)$ exceeds $\alpha$, in which case a valid offset $y$ cannot be found.)

Applying such a general procedure would be a better idea for a Beta distribution compared to using a Normal approximation, because Betas tend to be skewed (unless their parameters $a$ and $b$ are relatively similar).

For example, the orange region in the figure covers a $1-0.05$ HPD interval for a Beta$(11,4)$ distribution whose density is graphed. The dashed gray line shows the common value of the density at the endpoints.

Here is the R code that performed the calculation. It is written to be very general: if you can supply functions to compute $F$, $f$, and $F^{-1}$, it will work. (An example for Normal distributions has been commented out.)

offset <- function(x, alpha=0.05, F=pbeta, F.Inv=qbeta, ...) {
  q <- F(x, ...)
  y <- F.Inv(min(q + 1-alpha, 1), ...)
}
f <- function(x, alpha=0.05, F=pbeta, F.Inv=qbeta, f.dist=dbeta, ...) {
  y <- offset(x, alpha, F=F, F.Inv=F.Inv, ...)
  mapply(function(u,v) diff(f.dist(c(u,v), ...))^2, x, y) +
    (diff(F(c(x,y), ...)) - (1-alpha))^2
}
#
# Specify the problem.
#
alpha <- 0.05    # Level of the CI (between 0 and 1)
a <- 11; b <- 4  # Parameters
F <- pbeta       # Distribution function
F.Inv <- qbeta   # Inverse distribution function
f.dist <- dbeta  # Density function
x.min <- 0       # Minimum to consider
x.max <- 1       # Maximum to consider
# F <- pnorm
# F.Inv <- qnorm
# f.dist <- dnorm
# x.min <- -2
# x.max <- 5
#
# Find the solution.
#
x.0 <- qbeta(alpha/2, a, b)
x.lim <- qbeta(alpha, a, b)
sol <- optimize(function(x) f(x, alpha, F=F, F.Inv=F.Inv, f.dist=f.dist, a, b), 
                interval=c(x.min, x.lim), tol=1e-10)
x <- sol$minimum
y <- offset(x, alpha, F=F, F.Inv=F.Inv, a, b)
#
# Plot the solution.
#
u <- seq(x, y, length.out=1001)
v <- f.dist(u, a, b)
u <- c(u[1], u, u[length(u)])
v <- c(0, v, 0)
curve(f.dist(x, a, b), xlim=c(x.min, x.max), ylim=c(0,max(v)*1.2), n=1001, xlab="X", ylab="Density")
polygon(u, v, col="#f8e0a0", border=NA)
abline(h = f.dist(x, a, b), col="Gray", lty=2, lwd=2)
curve(f.dist(x, a, b), add=TRUE, lwd=2)

Best Answer

Related Solutions

Solved – Order statistic for beta distribution

Solved – Credible set for beta distribution

Related Question