Solved – When to use Bayesian Networks over other machine learning approaches

bayesian networkmachine learning

I expect there may be no definitive answer to this question. But I have used a number of machine learning algorithms in the past and am trying to learn about Bayesian Networks. I would like to understand under what circumstance, or for what types of problems would you choose to use Bayesian Network over other approaches?

Best Answer

Bayesian Networks (BN's) are generative models. Assume you have a set of inputs, $X$, and output $Y$. BN's allow you to learn the joint distribution $P(X,Y)$, as opposed to let's say logistic regression or Support Vector Machine, which model the conditional distribution $P(Y|X)$.

Learning the joint probability distribution (generative model) of data is more difficult than learning the conditional probability (discriminative models). However, the former provides a more versatile model where you can run queries such as $P(X_1|Y)$ or $P(X_1|X_2=A, X_3=B)$, etc. With the discriminative model, your sole aim is to learn $P(Y|X)$.

BN's utilize DAG's to prescribe the joint distribution. Hence they are graphical models.

Advantages:

  1. When you have a lot of missing data, e.g. in medicine, BN's can be very effective since modeling the joint distribution (i.e. your assertion on how the data was generated) reduces your dependency in having a fully observed dataset.

  2. When you want to model a domain in a way that is visually transparent, and also aims to capture $\text{cause} \to \text{effect}$ relationships, BN's can be very powerful. Note that the causality assumption in BN's is open to debate though.

  3. Learning the joint distribution is a difficult task, modeling it for discrete variables (through the calculation of conditional probability tables, i.e. CPT's) is substantially easier than trying to do the same for continuous variables though. So BN's are practically more common with discrete variables.

  4. BN's not only allow observational inference (as all machine learning models allow) but also causal interventions. This is a commonly neglected and underappreciated advantage of BN's and is related to counterfactual reasoning.

Related Question