[Math] How to calculate a posterior probability with a given Gaussian Mixture Model

machine learningMATLABnotationprobability distributionsstatistics

I'm building a GMM-based classifier in speech processing and I'm using GMM as a probabilistic scoring mechanism (therefore I don't intrinsically care about the underlying mixture components). For example, I have two classes of speech, neutral and stressed, each has its own trained GMM, and I classify an unknown speech by scoring its feature with the GMMs of the two classes and comparing the results against some decision threshold.

I already have an open-source EM/GMM matlab code to train my model (therefore I have the GMM parameters), but I don't know how exactly I'm supposed to calculate a posterior probability based on it.

I've also found the equation I need (from this paper), but I'm having a hard time translating it into matlab code (too bad my math has gotten rusty):

  1. In the second equation, what do the sigma notations mean? First one=square root of the determinant of the ith covariance matrix? Second one=summing from i to -1 of observation vector minus ith mean vector?

  2. What does (2*pi)^(D/2) mean? raising 2pi to the power of (D/2)? what if D is odd?

I wrote a silly matlab function of what I did so far:

function prob =GMMposterior(x,mu,covmat,weights)
% inputs: x, Dx1 observation vector;
%         mu, Dxn mean vector (n is the number of components)
%         covmat, DxDxn covariance matrix; 
%         weights, 1xn component weights. 

% output: prob, a posterior probability

d=size(x,1); % get the number of dimensions
w=size(weights,2); % get the number of compoments/weights
g=zeros(w,1);
for i=1:w
    g(i)=exp(-1/2*(x-mu(:,i))'/???)/((2*pi)^(round?(d/2)*sqrt?(det(covmat(:,:,i)));
end

prob=weights*g;
end

Best Answer

  1. First: Yes, the square root of the determinant of the $i$-th covariance matrix. Second: No, the inverse of the $i$-th covariance matrix.
  2. Yes, raising $2\pi$ to the $D/2$-th power. If $D$ is odd, this is a half integer; you still need to use $D/2$, without the rounding suggested in your code. $x^{D/2}=\sqrt{x^D}$.
Related Question