I spent the day learning about the bnlearn package in R only to discover that Bayesian models do not work with undirected graphs. I'm trying to learn about the Markov Random Field Network, and so far all I have been able to do is create the graphical structure using a graphical LASSO.
In directed graphs, there seems to be two stages: "structural learning" performed by some method, and then "parameter learning" performed by another method. My sense is that parameter learning tells you about the edge weights between each variable (feature) included in your model. My question is… so what? What do you do with a graph with edge weights?
If I have a dataset that is observations by features, and my graph's nodes are the features from this dataset (gleaned from the graphical LASSO trying to emulate the inverse of the covariance matrix), what can I learn from this? Can I compare cohorts of my data (separated by target class value) and assign some sort of p-value analysis with the nodes? I am confused, I think, about the big picture of graphical models.
Best Answer
The value and power of graphical models of conditional probabilistic relations is that they convey information about the causal structure and inferential structure of the system. For example, rain or sprinklers can cause wet sidewalks, but wet sidewalks cannot cause rain or sprinklers spraying. If your evidence is "rain" and "not sprinklers" you can infer certain things. If your evidence is "wet sidewalk" you can infer other things. What you can and can't infer is determined by the structure of the graph.
Your example of "a dataset that is observations by features" may or may not be well suited to a probabilistic graph. Adding in ideas/methods like "p-values" will only confuse you, I think. (P-values apply to frequentist hypothesis testing.) Metaphorically, you are combining apples, oranges, and armadillos.
I suggest that you view lectures by Judea Pearl (https://www.youtube.com/watch?v=zHjdd--W6o4, https://www.youtube.com/watch?v=IiXvpPyhMw8) plus tutorial lectures like these: https://www.youtube.com/watch?v=YvdpnqMRmfk, https://www.youtube.com/watch?v=Xhdpk9HZQuo.