How Gaussian Mixture Models Cluster Data

Gaussian mixture models (GMMs) are often used for data clustering. You can use GMMs to perform eitherhard簇ing or柔软的簇ing on query data.

To performhard聚类，GMM将查询数据点分配给了给定数据，该数据点指向最大化组件后验概率的多元正常组件。也就是说，鉴于合适的GMM，簇assigns query data to the component yielding the highest posterior probability. Hard clustering assigns a data point to exactly one cluster. For an example showing how to fit a GMM to data, cluster using the fitted model, and estimate component posterior probabilities, seeCluster Gaussian Mixture Data Using Hard Clustering。

此外，您可以使用GMM在数据上执行更灵活的聚类，称为柔软的（或者fuzzy）簇ing. Soft clustering methods assign a score to a data point for each cluster. The value of the score indicates the association strength of the data point to the cluster. As opposed to hard clustering methods, soft clustering methods are flexible because they can assign a data point to more than one cluster. When you perform GMM clustering, the score is the posterior probability. For an example of soft clustering with a GMM, see集群使用软Clusterin高斯混合数据g。

GMM clustering can accommodate clusters that have different sizes and correlation structures within them. Therefore, in certain applications,, GMM clustering can be more appropriate than methods such ask-means clustering. Like many clustering methods, GMM clustering requires you to specify the number of clusters before fitting the model. The number of clusters specifies the number of components in the GMM.

对于GMM，请遵循以下最佳实践：

Fit GMM with Different Covariance Options and Initial Conditions

此示例探讨了在执行GMM聚类时指定协方差结构和初始条件的不同选项的效果。

Load Fisher's iris data set. Consider clustering the sepal measurements, and visualize the data in 2-D using the sepal measurements.

loadfisheriris;X = meas(:,1:2); [n,p] = size(X); plot(X(:,1),X(:,2),'。'，，，，'MarkerSize'，15）;标题（'Fisher''s Iris Data Set'）；Xlabel（'萼片长度（cm）'）；ylabel（'萼片宽度（cm）'）；

The number of componentskin a GMM determines the number of subpopulations, or clusters. In this figure, it is difficult to determine if two, three, or perhaps more Gaussian components are appropriate. A GMM increases in complexity askincreases.

rng(3); k = 3;% Number of GMM componentsoptions = statset('MaxIter'，，，，1000);

Sigma = {'diagonal'，，，，'full'};% Options for covariance matrix typenSigma = numel(Sigma); SharedCovariance = {true,false};同一或非相互协方差矩阵的％指标SCtext = {'true'，，，，'false'};nsc = numel（sharedCovariance）;

d = 500;% Grid lengthx1 = linspace（min（x（：：，1））） -  2，max（x（：，1））+2，d）;x2 = linspace（min（x（：：，2）） -  2，max（x（：，2））+2，d）;[x1grid，x2grid] = meshgrid（x1，x2）;x0 = [x1grid（:) x2grid（:)];

threshold = sqrt(chi2inv(0.99,2)); count = 1;fori = 1:nSigmaforj = 1:nSC gmfit = fitgmdist(X,k,“协会型”，，，，Sigma{i},...“共享努力”，sharedCovariance {j}，'Options'，选项）;% Fitted GMMclusterx = cluster（gmfit，x）;% Cluster indexmahalDist = mahal(gmfit,X0);每个网格点到每个GMM组件的距离％% Draw ellipsoids over each GMM component and show clustering result.subplot(2,2,count); h1 = gscatter(X(:,1),X(:,2),clusterX); holdonform = 1:k idx = mahalDist(:,m)<=threshold; Color = h1(m).Color*0.75 - 0.5*(h1(m).Color - 1); h2 = plot(X0(idx,1),X0(idx,2),'。'，，，，'Color'，颜色，'MarkerSize'，1）;uistack（H2，'底部'）；end绘图（gmfit.mu（：，1），gmfit.mu（：，2），'kx'，，，，'LineWidth'，，，，2，，，，'MarkerSize'，10）标题（sprintf（Sprintf）（'Sigma是％s \ nsharedCovariance =％s'，sigma {i}，sctext {j}），'FontSize'，，，，8) legend(h1,{'1'，，，，'2'，，，，'3'}) holdoffcount = count + 1;endend

initialCond1 = [ones(n-8,1); [2; 2; 2; 2]; [3; 3; 3; 3]];% For the first GMMinitialCond2 = randsample(1:k,n,true);% For the second GMMinitialcond3 = randsample（1：k，n，true）;% For the third GMMinitialCond4 ='plus';第四个GMM的％簇0 = {initialCond1; initialCond2; initialCond3; initialCond4};

converged = nan(4,1);forj = 1:4 gmfit = fitgmdist(X,k,“协会型”，，，，'full'，，，，...“共享努力”，，，，false,'开始'，，，，簇0{j},...'Options'，选项）;clusterx = cluster（gmfit，x）;% Cluster indexmahalDist = mahal(gmfit,X0);每个网格点到每个GMM组件的距离％% Draw ellipsoids over each GMM component and show clustering result.subplot(2,2,j); h1 = gscatter(X(:,1),X(:,2),clusterX);每个网格点到每个GMM组件的距离％抓住on;nK = numel(unique(clusterX));form = 1：nk idx = mahaldist（：，m）<=阈值;颜色= H1（M）.Color*0.75 + -0.5*（H1（M）.Color -1）;h2 = plot（x0（idx，1），x0（idx，2），，'。'，，，，'Color'，颜色，'MarkerSize'，1）;uistack（H2，'底部'）；end绘图（gmfit.mu（：，1），gmfit.mu（：，2），'kx'，，，，'LineWidth'，，，，2，，，，'MarkerSize'，10）传奇（H1，{'1'，，，，'2'，，，，'3'}); holdoffconverged(j) = gmfit.Converged;百分比收敛指标end

sum(converged)

ANS = 4

何时正规化

Sometimes, during an iteration of the EM algorithm, a fitted covariance matrix can become ill conditioned, which means the likelihood is escaping to infinity. This problem can happen if one or more of the following conditions exist:

To overcome this problem, you can specify a small, positive number using the'RegularizationValue'名称值对参数。fitgmdist将这个数字添加到所有c的对角元素ovariance matrices, which ensures that all matrices are positive definite. Regularizing can reduce the maximal likelihood value.

Model Fit Statistics

In most applications, the number of componentskand appropriate covariance structure Σ are unknown. One way you can tune a GMM is by comparing information criteria. Two popular information criteria are the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC).

Both the AIC and BIC take the optimized, negative loglikelihood and then penalize it with the number of parameters in the model (the model complexity). However, the BIC penalizes for complexity more severely than the AIC. Therefore, the AIC tends to choose more complex models that might overfit, and the BIC tends to choose simpler models that might underfit. A good practice is to look at both criteria when evaluating a model. Lower AIC or BIC values indicate better fitting models. Also, ensure that your choices forkand the covariance matrix structure are appropriate for your application.fitgmdiststores the AIC and BIC of fittedgmdistribution属性中的模型对象AICandBIC。You can access these properties by using dot notation. For an example showing how to choose the appropriate parameters, seeTune Gaussian Mixture Models。