[Math] Is a mixture of two uniform distributions more complex than a single distribution

probabilityprobability distributions

I'm a psychologist studying perception of visual ensembles (e.g., lots of lines with different orientations drawn on a screen) that have different underlying probability distributions. One of the reviewer's for our paper has asked us to justify a statement that seems intuitively correct to me but I wasn't able to find a proper reference. The statement in question says that a mixture of two uniform distributions is more complex than a normal or a uniform one. The mixture distribution here consists of two non-intersecting uniform distributions with equal ranges but different means. Intuitively it seems to me that it should be more complex as its probability density function has more parameters than the functions of a single uniform or a normal distribution hence it could be said that it has lower "description length".

Am I correct in saying that this mixture distribution is more complex? And if so, could you please provide any reference supporting this?

Best Answer

in one of your comments you mentioned the paper "Measuring the complexity of continuous distributions". In this paper Complexity is defined as a function of Shannon's entropy. Shannon's entropy provide a measure of the average uncertainty of a system given a probability distribution. Thus, this Complexity determines the "balance" between emergence of new patterns, and the self-organization of the system.

Intuitively, Emergence can be understood as the uniformization of a probability distribution (for instance, the white noise which has a uniform distribution has the highest Emergence). Self-organization can be understood as the concentration of probability around a specific state(s) of the probability distribution (a delta Dirac has the highest self-organization, thus, the lowest emergence since there is no change). Then, a distribution with high complexity has one or few states which concentrate a large proportion of the probability, and many others with very low probability.

Thus, using these measures, it is not conclusive if a mixture of distributions will be more complex than only 1 distribution. For instance, compare a power-law distribution with the second mixture provided by @BruceET. Given a suitable x_min value, the power-law will surely display more complexity than the mixture of normals, since the latter is more equiprobable than the former.

Best regards, Guillermo