Is it possible to apply the usual MLE procedure to the triangle distribution? – I am trying but I seem to be blocked at one step or another in the math by the way the distribution is defined. I am trying to use the fact that I know the number of samples above and below c (without knowing c): these 2 numbers are cn and (1-c)n, if n is the total number of samples. However, that does not seem to help in the derivation. The moment of moments gives an estimator for c without much problem. What is the exact nature of the obstruction to MLE here (if indeed there is one)?
More details:
Let's consider $c$ in $[0,1]$ and the distribution defined on $[0,1]$ by:
$f(x;c) = \frac{2x}{c}$ if x < c
$f(x;c) = \frac{2(1-x)}{(1-c)}$ if c <= x
Let's take an $n$ i.i.d samples $\{x_{i}\}$ from this distribution form the log-likelihood of c given this sample:
$\hat{l}(c | \{x_{i}\}) = \sum_{i=1}^{n}ln(f(x_{i}|c))$
I am then trying to use the fact that given the form of $f$, we know that $cn$ samples will fall below the (unknown) $c$, and $(1-c)n$ will fall above $c$. IMHO, this allows to decompose the summation in the expression of the log-likelihood thus:
$\hat{l}(c | \{x_{i}\}) = \sum_{i=1}^{cn}ln\frac{2 x_{i}}{c} + \sum_{i=1}^{(1-c)n}ln\frac{2(1-x_{i})}{1-c}$
Here, I am unsure how to proceed. MLE will involve taking a derivative w.r.t. $c$ of the log-likelihood, but I have $c$ as the upper bound of the summation, which seems to block that. I could try with another form of the log-likelihood, using indicator functions:
$\hat{l}(c | \{x_{i}\}) = \sum_{i=1}^{n}\{x_{i}<c\}ln\frac{2 x_{i}}{c} + \sum_{i=1}^{n}\{c<=x_{i}\}ln\frac{2(1-x_{i})}{1-c}$
But deriving the indicators doesn't seem easy either, although Dirac deltas could allow to continue (while still having indicators, since we need to derive products).
So, here I am blocked in MLE. Any idea?
Best Answer
Certainly! Though there are some oddities to deal with, it's possible to compute MLEs in this case.
However, if by 'the usual procedure' you mean 'take derivatives of the log-likelihood and set it equal to zero', then maybe not.
Have you tried drawing the likelihood?
--
Followup after clarification of question:
The question about drawing the likelihood was not idle commentary, but central to the issue.
No. MLE involves finding the argmax of a function. That only involves finding the zeros of a derivative under certain conditions... which don't hold here. At best, if you manage to do that you'll identify a few local minima.
As my earlier question suggested, look at the likelihood.
Here's a sample, $y$ of 10 observations from a triangular distribution on (0,1):
Here's the likelihood and log-likelihood functions for $c$ on that data:
The grey lines mark the data values (I should probably have generated a new sample to get better separation of the values). The black dots mark the likelihood / log-likelihood of those values.
Here's a zoom in near the maximum of the likelihood, to see more detail:
As you can see from the likelihood, at many of the order statistics, the likelihood function has sharp 'corners' - points where the derivative doesn't exist (which is no surprise - the original pdf has a corner and we're taking a product of pdfs). This (that there are cusps at order statistics) is the case with the triangular distribution, and the maximum always occurs at one of the order statistics. (That cusps occur at order statistics isn't unique to the triangular distributions; for example the Laplace density has a corner and as a result the likelihood for its center has one at each order statistic.)
As it happens in my sample, the maximum occurs as the fourth order statistic, 0.3780912
So to find the MLE of $c$ on (0,1), just find the likelihood at each observation. The one with the biggest likelihood is the MLE of $c$.
A useful reference is chapter 1 of "Beyond Beta" by Johan van Dorp and Samuel Kotz. As it happens, Chapter 1 is a free 'sample' chapter for the book - you can download it here.
There's a lovely little paper by Eddie Oliver on this issue with the triangular distribution, I think in American Statistician (which makes basically the same points; I think it was in a Teacher's Corner). If I can manage to locate it I'll give it as a reference.
Edit: here it is:
E. H. Oliver (1972), A Maximum Likelihood Oddity,
The American Statistician, Vol 26, Issue 3, June, p43-44
(publisher link)
If you can easily get hold of it, it's worth a look, but that Dorp and Kotz chapter covers most of the relevant issues so it's not crucial.
By way of followup on the question in comments - even if you could find some way of 'smoothing off' the corners, you'd still have to deal with the fact that you can get multiple local maxima:
It might, however, be possible to find estimators that have very good properties (better than method of moments), which you can write down easily. But ML on the triangular on (0,1) is a few lines of code.
If it's a matter of huge amounts of data, that, too, can be dealt with, but would be another question, I think. For example, not every data point can be a maximum, which reduces the work, and there are some other savings that can be made.