Solved – Why do we need density in estimation and cumulative distribution in transformation

cumulative distribution functiondensity functionempirical-cumulative-distr-fnmaximum likelihood

In statistic, when we want to find the estimation of the model parameters, do we only work with the density function?

Why do we need to work with density function instead of Cumulative distribution function?

Also, sometimes we need to work with cumulative distribution function. For example, if we need to transform the margins to be uniform margins we can use cumulative distribution function, or sometimes the empirical cumulative distribution function.

Why we do not use density in transformation and use it in estimation?

What are the benefits of density function over the Cumulative distribution function and ecdf and vice verse?

Best Answer

I guess that what you mean is maximum likelihood estimation scenario, where we consider some value $\hat \theta$ as the best guess for $\theta$ if it maximizes the likelihood function:

$$ L(\theta) = \prod_{i=1}^n f(x_i|\theta) $$

Imagine a simple model, where we want to estimate the mean of normal distribution with known standard deviation $\sigma$, say that instead probability density function you used cumulative density function and tried to maximize $\prod_{i=1}^n F(x_i|\mu)$, then no matter of what your data is the perfect choice for $\mu$ is $-\infty$ since $\Pr(X -\infty \le x) = 1$ for any $x$, so the function reaches it's maximum at such value.

Same said visually, look at the following plot, it shows standard normal density function (black) and standard normal cumulative distribution function (gray). The most probable value is $X=\mu=0$, where the density distribution peaks. On another hand, the density at $X=\infty$ is zero, while $\Pr(X=\infty)=1$. Cumulative distribution function does not let you to find the peak by looking at its maximum.

Related Solutions

How to Calculate Cumulative Distribution in R – Step-by-Step Guide

The ecdf function applied to a data sample returns a function representing the empirical cumulative distribution function. For example:

> X = rnorm(100) # X is a sample of 100 normally distributed random variables
> P = ecdf(X)    # P is a function giving the empirical CDF of X
> P(0.0)         # This returns the empirical CDF at zero (should be close to 0.5)
[1] 0.52
> plot(P)        # Draws a plot of the empirical CDF (see below)

enter image description here

If you want to have an object representing the empirical CDF evaluated at specific values (rather than as a function object) then you can do

> z = seq(-3, 3, by=0.01) # The values at which we want to evaluate the empirical CDF
> p = P(z)                # p now stores the empirical CDF evaluated at the values in z

Note that p contains at most the same amount of information as P (and possibly it contains less) which in turn contains the same amount of information as X.

Solved – Why maximum likelihood estimation use the product of pdfs rather than cdfs

How can a CDF be used to rank two possible parametrizations for a model? It is a cumulative probability, so it can only tell us the probability of obtaining such a result or a lower value given a probability model. If we took $\theta$ to predict the smallest possible outcomes, the CDF is nearly 1 at every observation and this would be the most "likely" in the sense that "yup, if the mean height were truly -99 I am very confident that repeating my sample would produce values smaller than the ones I observed".

We could balance the left cumulative probability with the right cumulative probability. Consider the converse in our calculation: a median unbiased estimator satisfies:

$$P(X < \theta) = P(X > \theta)$$

Here the best value of $\theta$ is the one for which $X$ is equally likely to be greater or less than it's predicted value (assuming $\theta$ is a mean here). But that certainly doesn't correspond with our idea of being able to rank alternate parameterizations as more likely for a particular sample.

Perhaps, on the other hand you wanted to be sure $X$ was very probable in a small interval of the value, that is maximize this probability:

$$P(\theta - d < X < \theta + d)/d = \left(F(X+d) - F(X-d)\right)/d$$

But how big should $d$ be? Well if $d$ is taken to be arbitrarily small:

$$\lim_{d \rightarrow 0} \left(F(X+d) - F(X-d)\right)/d = f(X)$$

And you get the density. It is the instantaneous probability function that best characterizes the likelihood of a specific observation under a parametrization.

Best Answer

Related Solutions

How to Calculate Cumulative Distribution in R – Step-by-Step Guide

Solved – Why maximum likelihood estimation use the product of pdfs rather than cdfs

Related Question