For the classical RBM: $P(\tilde{h}|\tilde{x})=\sigma\left(\tilde{b}+W\tilde{x}\right)
a$ and $P(\tilde{x}|\tilde{h})=\sigma\left(\tilde{c}+W^{T}\tilde{h}\right)$
for for hidden layer $\tilde{h}$ and visible layer $\tilde{x}$ , and weight matrix $W$ and bias vectors $\tilde{b}$ and $\tilde{c}$
Is the correct (/preferred) name for this:
- a Binary-Binary RBM,
- a Bernoulli-Bernoulli RBM,
- a Binary RBM,
- a Bernoulli RBM
I see Binary RBM most often, in papers — if it is specified at all, which it usually isn't, this type of RBM is the default.
Bernoulli-Bernoulli RBM makes the most sense to me,
as the elements in the visible and in the hidden layers are assumed to be Bernoulli distributed.
Which means the take Binary values.
Bernoulli-Bernoulli also works better if we have
Gaussian-Bernoulli RBMs also being talked about, as this speaks of the distrobutions of each layer.
However I read papers which say Gaussian-Binary RBM, which to me is not right as Binary is not a distribution, but Gaussian is.
What is preferred?
Best Answer
I don't think it matters, since it is mostly clear from context. The hiddens are nearly always binary. Bernoulli and binary mean the same thing. If another distribution is mentioned, e.g. as in "Gaussian RBM" this refers to an RBM with Gaussian visibles.