Implements an infinite Gaussian Mixture Model using a Dirichlet Process Mixture Model (DPMM) and Gibbs sampling. More...

#include <cluster.h>

Collaboration diagram for cmp::cluster::DirichletProcessMixtureModel:

[legend]

Classes
struct	Cluster

Public Member Functions
	DirichletProcessMixtureModel (double alpha, const cmp::distribution::GammaDistribution &alphaPrior, const cmp::distribution::NormalInverseWishartDistribution &hyper, unsigned int seed=12345)
	Constructs a new DirichletProcessMixtureModel object.

void	condition (const Eigen::Ref< const Eigen::MatrixXd > &data, const Eigen::VectorXs &init_labels)
	Conditions the DPMM model on the given dataset with initial cluster labels.

void	step ()
	Performs one complete sweep of collapsed Gibbs sampling over all points.

void	remapLabels ()
	Remaps cluster labels to be contiguous integers starting from 0.

Eigen::VectorXs	getLabels () const
	Gets the current cluster label assignments vector.

size_t	nClusters () const
	Gets the current number of active clusters.

double	getAlpha () const
	Gets the current concentration parameter alpha.

Private Member Functions
void	init (const Eigen::VectorXs &init_labels)
	Initializes cluster counts, sums, and sufficient statistics.

void	removePointFromCluster (const size_t &pointIndex, const size_t &clusterID)
	Removes a point from the sufficient statistics of a specified cluster.

void	addPointToCluster (const size_t &pointIndex, const size_t &clusterID)
	Adds a point to the sufficient statistics of a specified cluster.

double	logNIWPosterior (const Eigen::VectorXd &x, const Cluster &c) const
	Computes the log posterior probability of a point under a cluster's NIW predictive distribution.

void	updateAlpha ()
	Updates the concentration parameter alpha via auxiliary variable sampling.

Private Attributes
std::vector< double >	logp_workspace_
	Temporary buffer for cluster log-probabilities.

std::vector< double >	probs_workspace_
	Temporary buffer for cluster probability weights.

std::vector< size_t >	clusterIDs_workspace_
	Temporary buffer for cluster IDs.

Eigen::MatrixXd	xObs_
	Observed data points matrix (N x D).

Eigen::VectorXs	labels_
	Current cluster assignment label vector.

size_t	nPoints_ {0}
	Number of observation points.

int	dim_ {0}
	Dimensionality of the feature space.

std::vector< size_t >	clusterIDs_
	List of active cluster IDs.

cmp::distribution::NormalInverseWishartDistribution	hyper_
	Hyperprior distribution parameters for clusters.

cmp::distribution::GammaDistribution	alphaPrior_
	Prior distribution parameters for concentration parameter.

double	alpha_ {1.0}
	Concentration parameter alpha for the Dirichlet Process.

std::map< size_t, Cluster >	clusters_
	Map from cluster ID to sufficient statistics cluster structure.

size_t	nextClusterId_
	Counter to generate unique new cluster IDs.

std::default_random_engine	rng_
	Pseudo-random number generator engine.

std::uniform_real_distribution< double >	distU_ {0.0, 1.0}
	Uniform real generator for rejection sampler.

Detailed Description

Implements an infinite Gaussian Mixture Model using a Dirichlet Process Mixture Model (DPMM) and Gibbs sampling.

Mathematical Foundations

The Dirichlet Process Mixture Model (DPMM) assumes an infinite number of mixture components, allowing the model to adaptively determine the number of clusters from the data. The generative process for observations \(\mathbf{x}_i \in \mathbb{R}^d\) is:

\[ \mathbf{x}_i | z_i, \{\boldsymbol{\mu}_k, \boldsymbol{\Sigma}_k\} \sim \mathcal{N}(\boldsymbol{\mu}_{z_i}, \boldsymbol{\Sigma}_{z_i}) \]

\[ z_i | \mathbf{w} \sim \mathrm{Categorical}(\mathbf{w}) \]

\[ \mathbf{w} \sim \mathrm{GEM}(\alpha) \quad \text{(Stick-breaking construction)} \]

\[ (\boldsymbol{\mu}_k, \boldsymbol{\Sigma}_k) \sim \mathrm{NIW}(\boldsymbol{\mu}_0, \kappa_0, \boldsymbol{\Lambda}_0, \nu_0) \]

where \(\mathrm{NIW}\) is the Normal-Inverse-Wishart conjugate prior, and \(\alpha > 0\) is the concentration parameter.

The NIW prior is defined as:

\[ \boldsymbol{\Sigma} \sim \mathrm{Inv-Wishart}(\boldsymbol{\Lambda}_0, \nu_0), \quad \boldsymbol{\mu} | \boldsymbol{\Sigma} \sim \mathcal{N}\left(\boldsymbol{\mu}_0, \frac{1}{\kappa_0} \boldsymbol{\Sigma}\right) \]

Implementation Algorithms (Collapsed Gibbs Sampling)

The algorithm integrates out the parameters \(\{\boldsymbol{\mu}_k, \boldsymbol{\Sigma}_k\}\), sampling only the cluster indicators \(z_i\):

Existing Cluster Probability: The probability that data point \(\mathbf{x}_i\) belongs to an existing cluster \(k\) is:
\[ P(z_i = k | \mathbf{z}_{-i}, \mathbf{x}_i) \propto n_k^{-i} \cdot t_{\nu_n}\left(\mathbf{x}_i ; \boldsymbol{\mu}_n, \frac{\kappa_n + 1}{\kappa_n(\nu_n - d + 1)} \boldsymbol{\Lambda}_n\right) \]
where \(n_k^{-i}\) is the cluster size excluding \(\mathbf{x}_i\), and \(t_{\nu}\) is the multivariate Student-t distribution representing the posterior predictive distribution.
New Cluster Probability: The probability that \(\mathbf{x}_i\) spawns a new cluster is:
\[ P(z_i = \text{new} | \mathbf{z}_{-i}, \mathbf{x}_i) \propto \alpha \cdot t_{\nu_0}\left(\mathbf{x}_i ; \boldsymbol{\mu}_0, \frac{\kappa_0 + 1}{\kappa_0(\nu_0 - d + 1)} \boldsymbol{\Lambda}_0\right) \]
NIW Posterior Updates: For each cluster \(k\), the sufficient statistics are updated as:
\[ \kappa_n = \kappa_0 + n_k, \quad \nu_n = \nu_0 + n_k, \quad \boldsymbol{\mu}_n = \frac{\kappa_0 \boldsymbol{\mu}_0 + n_k \bar{\mathbf{x}}}{\kappa_n} \]

\[ \boldsymbol{\Lambda}_n = \boldsymbol{\Lambda}_0 + \mathbf{C} + \frac{\kappa_0 n_k}{\kappa_n} (\bar{\mathbf{x}} - \boldsymbol{\mu}_0)(\bar{\mathbf{x}} - \boldsymbol{\mu}_0)^T \]
where \(\bar{\mathbf{x}}\) is the sample mean, and \(\mathbf{C} = \sum_{j \in \text{cluster}} (\mathbf{x}_j - \bar{\mathbf{x}})(\mathbf{x}_j - \bar{\mathbf{x}})^T\).

Constraints & Invariants

Degrees of Freedom: Prior degrees of freedom must satisfy \(\nu_0 > d - 1\) to guarantee prior scale matrix normalization.
Positive Definiteness: The prior scale matrix \(\boldsymbol{\Lambda}_0\) must be symmetric positive-definite.

Constructor & Destructor Documentation

◆ DirichletProcessMixtureModel()

cmp::cluster::DirichletProcessMixtureModel::DirichletProcessMixtureModel	(	double	alpha,
		const cmp::distribution::GammaDistribution &	alphaPrior,
		const cmp::distribution::NormalInverseWishartDistribution &	hyper,
		unsigned int	seed = `12345`
	)

Constructs a new DirichletProcessMixtureModel object.

Parameters

alpha	Initial concentration parameter for the Dirichlet Process.
alphaPrior	Gamma prior distribution parameters for concentration updates.
hyper	Normal-Inverse-Wishart hyperprior parameters for clusters.
seed	Seed value for the random number generator.

Member Function Documentation

◆ addPointToCluster()

void cmp::cluster::DirichletProcessMixtureModel::addPointToCluster	(	const size_t &	pointIndex,
		const size_t &	clusterID
	)

private

Adds a point to the sufficient statistics of a specified cluster.

Parameters

pointIndex	Index of the data point.
clusterID	ID of the cluster.

◆ condition()

void cmp::cluster::DirichletProcessMixtureModel::condition	(	const Eigen::Ref< const Eigen::MatrixXd > &	data,
		const Eigen::VectorXs &	init_labels
	)

Conditions the DPMM model on the given dataset with initial cluster labels.

Parameters

data	N x D matrix of observed points.
init_labels	Initial cluster assignment vector of size N.

◆ getAlpha()

double cmp::cluster::DirichletProcessMixtureModel::getAlpha ( ) const

inline

Gets the current concentration parameter alpha.

Returns: Value of alpha.

◆ getLabels()

Eigen::VectorXs cmp::cluster::DirichletProcessMixtureModel::getLabels ( ) const

inline

Gets the current cluster label assignments vector.

Returns: Eigen::VectorXs of cluster labels.

◆ init()

void cmp::cluster::DirichletProcessMixtureModel::init ( const Eigen::VectorXs & init_labels )

private

Initializes cluster counts, sums, and sufficient statistics.

Parameters

init_labels Initial cluster labels.

◆ logNIWPosterior()

double cmp::cluster::DirichletProcessMixtureModel::logNIWPosterior	(	const Eigen::VectorXd &	x,
		const Cluster &	c
	)		const

private

Computes the log posterior probability of a point under a cluster's NIW predictive distribution.

Parameters

x	The point vector.
c	The cluster structure.

Returns: The log posterior probability.

◆ nClusters()

size_t cmp::cluster::DirichletProcessMixtureModel::nClusters ( ) const

inline

Gets the current number of active clusters.

Returns: Number of clusters.

◆ remapLabels()

void cmp::cluster::DirichletProcessMixtureModel::remapLabels ( )

Remaps cluster labels to be contiguous integers starting from 0.

◆ removePointFromCluster()

void cmp::cluster::DirichletProcessMixtureModel::removePointFromCluster	(	const size_t &	pointIndex,
		const size_t &	clusterID
	)

private

Removes a point from the sufficient statistics of a specified cluster.

Parameters

pointIndex	Index of the data point.
clusterID	ID of the cluster.

◆ step()

void cmp::cluster::DirichletProcessMixtureModel::step ( )

Performs one complete sweep of collapsed Gibbs sampling over all points.

◆ updateAlpha()

void cmp::cluster::DirichletProcessMixtureModel::updateAlpha ( )

inlineprivate

Updates the concentration parameter alpha via auxiliary variable sampling.

Member Data Documentation

◆ alpha_

double cmp::cluster::DirichletProcessMixtureModel::alpha_ {1.0}

private

Concentration parameter alpha for the Dirichlet Process.

◆ alphaPrior_

cmp::distribution::GammaDistribution cmp::cluster::DirichletProcessMixtureModel::alphaPrior_

private

Prior distribution parameters for concentration parameter.

◆ clusterIDs_

std::vector<size_t> cmp::cluster::DirichletProcessMixtureModel::clusterIDs_

private

List of active cluster IDs.

◆ clusterIDs_workspace_

std::vector<size_t> cmp::cluster::DirichletProcessMixtureModel::clusterIDs_workspace_

private

Temporary buffer for cluster IDs.

◆ clusters_

std::map<size_t, Cluster> cmp::cluster::DirichletProcessMixtureModel::clusters_

private

Map from cluster ID to sufficient statistics cluster structure.

◆ dim_

int cmp::cluster::DirichletProcessMixtureModel::dim_ {0}

private

Dimensionality of the feature space.

◆ distU_

std::uniform_real_distribution<double> cmp::cluster::DirichletProcessMixtureModel::distU_ {0.0, 1.0}

private

Uniform real generator for rejection sampler.

◆ hyper_

cmp::distribution::NormalInverseWishartDistribution cmp::cluster::DirichletProcessMixtureModel::hyper_

private

Hyperprior distribution parameters for clusters.

◆ labels_

Eigen::VectorXs cmp::cluster::DirichletProcessMixtureModel::labels_

private

Current cluster assignment label vector.

◆ logp_workspace_

std::vector<double> cmp::cluster::DirichletProcessMixtureModel::logp_workspace_

private

Temporary buffer for cluster log-probabilities.

◆ nextClusterId_

size_t cmp::cluster::DirichletProcessMixtureModel::nextClusterId_

private

Counter to generate unique new cluster IDs.

◆ nPoints_

size_t cmp::cluster::DirichletProcessMixtureModel::nPoints_ {0}

private

Number of observation points.

◆ probs_workspace_

std::vector<double> cmp::cluster::DirichletProcessMixtureModel::probs_workspace_

private

Temporary buffer for cluster probability weights.

◆ rng_

std::default_random_engine cmp::cluster::DirichletProcessMixtureModel::rng_

private

Pseudo-random number generator engine.

◆ xObs_

Eigen::MatrixXd cmp::cluster::DirichletProcessMixtureModel::xObs_

private

Observed data points matrix (N x D).

The documentation for this class was generated from the following files:

include/cluster.h
src/cluster.cpp

Classes

Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

Mathematical Foundations

Implementation Algorithms (Collapsed Gibbs Sampling)

Constraints & Invariants

Constructor & Destructor Documentation

◆ DirichletProcessMixtureModel()

Member Function Documentation

◆ addPointToCluster()

◆ condition()

◆ getAlpha()

◆ getLabels()

◆ init()

◆ logNIWPosterior()

◆ nClusters()

◆ remapLabels()

◆ removePointFromCluster()

◆ step()

◆ updateAlpha()

Member Data Documentation

◆ alpha_

◆ alphaPrior_

◆ clusterIDs_

◆ clusterIDs_workspace_

◆ clusters_

◆ dim_

◆ distU_

◆ hyper_

◆ labels_

◆ logp_workspace_

◆ nextClusterId_

◆ nPoints_

◆ probs_workspace_

◆ rng_

◆ xObs_