Probability – What Exactly Does the Term “Inverse Probability” Mean?

bayesianhistoryprobabilityterminology

I keep seeing the term "inverse probability" mentioned in passing albeit without any explanation.

I know it has to do with Bayesian inference, but what exactly do we mean by inverting a probability?

My thinking at the moment is that in "normal" probability we take outcomes of random experiments and try to estimate probabilities based on the outcomes of these experiments whereas in "inverse probability" we're interested in going from a probability (a prior for an unknown quantity) to knowing the "outcome of an experiment", the experiment being finding out the value of an unknown quantity (i.e. via the posterior, and maybe finding the MAP hypothesis).

That is, in "conventional probability" we go from outcomes of an experiment to probability vs. in inverse probability we go the other way: we go from a prior to uncovering the outcome of an experiment.

Best Answer

"Inverse probability" is a rather old-fashioned way of referring to Bayesian inference; when it's used nowadays it's usually as a nod to history. De Morgan (1838), An Essay on Probabilities, Ch. 3 "On Inverse Probabilities", explains it nicely:

In the preceding chapter, we have calculated the chances of an event, knowing the circumstances under which it is to happen or fail. We are now to place ourselves in an inverted position: we know the event, and ask what is the probability which results from the event in favour of of any set of circumstances under which the same might have happened.

An example follows using Bayes' Theorem.

I'm not sure that the term mightn't have at some point encompassed putative or proposed non-Bayesian, priorless, methods of getting from $f(y|\theta)$ to $p(\theta|y)$ (in @Christopher Hanck's notation); but at any rate Fisher was clearly distinguishing between "inverse probability" & his methods—maximum likelihood, fiducial inference—by the 1930's. It also strikes me that several early-20th-Century writers seem to view the use of what we now call uninformative/ignorance/reference priors as part & parcel of the "inverse probability" method, or even of "Bayes' Theorem".

† Fisher (1930), Math. Proc. Camb. Philos. Soc., 26, p 528, "Inverse probability", clearly distinguishes, perhaps for the first time, between Bayesian inference from flat "ignorance" priors ("the inverse argument proper"), the unexceptionable application of Bayes' Theorem when the prior describes aleatory probabilities ("not inverse probability strictly speaking"), & his fiducial argument.

‡ For example, Pearson (1907), Phil. Mag., p365, "On the influence of past experience on future expectation", conflates Bayes' Theorem with the "equal distribution of ignorance".

Related Question