Classifier that uses Kernel Density Estimation (KDE) for classification. More...

#include <classifier.h>

Inheritance diagram for cmp::classifier::KDE:

Collaboration diagram for cmp::classifier::KDE:

Public Member Functions
	KDE ()
	Kernel object for KDE.

void	set (std::shared_ptr< kernel::Kernel > kernel, std::shared_ptr< cmp::kernel::Bandwidth > bandwidth)
	Set the kernel and bandwidth for the KDE. This method allows you to set the kernel and bandwidth objects that will be used for the KDE.

void	condition (const Eigen::Ref< const Eigen::MatrixXd > &xObs, const Eigen::Ref< const Eigen::VectorXs > &labels)
	Condition the KDE with observations and labels. This method sets the observations and labels for the KDE. It initializes the number of observations, dimensions, and classes based on the input data.

double	density (const Eigen::Ref< const Eigen::VectorXd > &x, const size_t &classLabel) const
	Get the density of a class at a given point.

std::vector< double >	predictProbabilities (const Eigen::Ref< const Eigen::VectorXd > &x) const override
	Predict the class probabilities at a given point.

size_t	predict (const Eigen::Ref< const Eigen::VectorXd > &x) const override
	Predict the class label at a given point.

double	objectiveFunctionCV (const method &method, const cmp::statistics::KFold &kf) const
	Objective function for cross-validation. This function computes the objective value for cross-validation, to be maximized.

double	objectiveFunctionLOO () const
	Objective function for efficient leave-one-out optimization. This function computes mean LOO log-probability over all observations, to be maximized.

void	fit (const Eigen::Ref< const Eigen::MatrixXd > &xObs, const Eigen::Ref< const Eigen::VectorXs > &labels, cmp::statistics::KFold kf, const double &minBw, const double &maxBw, const method method=CV_PROB_SCORE, nlopt::algorithm algo=nlopt::LN_SBPLX, double ftol_rel=1e-4, std::vector< bool > logScaleFlags={})
	Fit the KDE using cross-validation to optimize the bandwidth. This method uses the specified optimization algorithm to find the optimal bandwidth that maximizes the cross-validation score.

void	fitLOO (const Eigen::Ref< const Eigen::MatrixXd > &xObs, const Eigen::Ref< const Eigen::VectorXs > &labels, const double &minBw, const double &maxBw, nlopt::algorithm algo=nlopt::LN_SBPLX, double ftol_rel=1e-4, std::vector< bool > logScaleFlags={})
	Fit the KDE by maximizing an efficient leave-one-out objective. This method optimizes bandwidth using mean LOO log-probability over all observations.

Public Member Functions inherited from cmp::classifier::Classifier
virtual	~Classifier ()=default

Private Attributes
Eigen::MatrixXd	xObs_ = Eigen::MatrixXd(0, 0)

Eigen::VectorXs	labels_ = Eigen::VectorXs(0)
	Observations.

Eigen::VectorXs	classCounts_
	Labels for the observations.

size_t	nObs_ = 0
	Count of observations per class.

size_t	dimX_ = 0
	Input features dimensionality.

size_t	nClasses_ = 1
	Total number of distinct class labels.

std::shared_ptr< cmp::kernel::Bandwidth >	bandwidth_ {nullptr}

std::shared_ptr< kernel::Kernel >	kernel_ {nullptr}
	Bandwidth object for KDE.

Detailed Description

Classifier that uses Kernel Density Estimation (KDE) for classification.

Mathematical Formulation Estimates the conditional probability density function for class \(c\) at point \(\mathbf{x} \in \mathbb{R}^D\) using kernel density estimation:

\[ p(\mathbf{x} \mid y = c) = \frac{1}{N_c} \sum_{i \in \mathcal{D}_c} \prod_{d=1}^D \frac{1}{h_{c,d}} K\left(\frac{x_d - x_{i,d}}{h_{c,d}}\right) \]

where \(N_c\) is the number of samples in class \(c\), \(h_{c,d}\) is the bandwidth for dimension \(d\) in class \(c\), and \(K\) is a univariate kernel. The class posterior probability is obtained using Bayes' rule:

\[ P(y = c \mid \mathbf{x}) = \frac{p(\mathbf{x} \mid y = c) P(y = c)}{\sum_{j=0}^{C-1} p(\mathbf{x} \mid y = j) P(y = j)} \]

where \(P(y = c) = \frac{N_c}{N}\) represents the empirical prior probability of class \(c\).

Implementation Algorithm

condition() partitions training samples by class label and computes class priors \(P(y=c)\).
predictProbabilities() computes the kernel density \(p(\mathbf{x} \mid y = c)\) for each class, computes the product of density and prior, and normalizes the results.
fit() optimizes the bandwidth vector \(\mathbf{h}\) by maximizing K-fold cross-validation likelihood or leave-one-out (LOO) log-probability using NLopt optimization routines.

Constructor & Destructor Documentation

◆ KDE()

cmp::classifier::KDE::KDE ( )

inline

Kernel object for KDE.

Member Function Documentation

◆ condition()

void cmp::classifier::KDE::condition	(	const Eigen::Ref< const Eigen::MatrixXd > &	xObs,
		const Eigen::Ref< const Eigen::VectorXs > &	labels
	)

inline

Condition the KDE with observations and labels. This method sets the observations and labels for the KDE. It initializes the number of observations, dimensions, and classes based on the input data.

Parameters

xObs	A matrix of observations where each row is an observation and each column is a feature.
labels	A vector of labels corresponding to the observations.

Exceptions

std::runtime_error if the number of observations and labels do not match.

◆ density()

double cmp::classifier::KDE::density	(	const Eigen::Ref< const Eigen::VectorXd > &	x,
		const size_t &	classLabel
	)		const

Get the density of a class at a given point.

Parameters

x	The point at which to evaluate the density.
classLabel	The label of the class for which to evaluate the density.

Returns: The density of the class at the given point.

Functions for the KDE classifier

◆ fit()

void cmp::classifier::KDE::fit	(	const Eigen::Ref< const Eigen::MatrixXd > &	xObs,
		const Eigen::Ref< const Eigen::VectorXs > &	labels,
		cmp::statistics::KFold	kf,
		const double &	minBw,
		const double &	maxBw,
		const method	method = `CV_PROB_SCORE`,
		nlopt::algorithm	algo = `nlopt::LN_SBPLX`,
		double	ftol_rel = `1e-4`,
		std::vector< bool >	logScaleFlags = `{}`
	)

Fit the KDE using cross-validation to optimize the bandwidth. This method uses the specified optimization algorithm to find the optimal bandwidth that maximizes the cross-validation score.

Parameters

xObs	A matrix of observations where each row is an observation and each column is a feature.
labels	A vector of labels corresponding to the observations.
kf	A KFold object containing the cross-validation splits.
minBw	The minimum bandwidth value for optimization.
maxBw	The maximum bandwidth value for optimization.
method	The method to use for cross-validation (CV_SCORE or CV_PROB_SCORE).
algo	The optimization algorithm to use (default is nlopt::LN_SBPLX).
ftol_rel	The relative tolerance for the optimization algorithm (default is 1e-4).
logScaleFlags	A vector of booleans indicating whether to use log-scaling for each bandwidth parameter (default is all false).

◆ fitLOO()

void cmp::classifier::KDE::fitLOO	(	const Eigen::Ref< const Eigen::MatrixXd > &	xObs,
		const Eigen::Ref< const Eigen::VectorXs > &	labels,
		const double &	minBw,
		const double &	maxBw,
		nlopt::algorithm	algo = `nlopt::LN_SBPLX`,
		double	ftol_rel = `1e-4`,
		std::vector< bool >	logScaleFlags = `{}`
	)

Fit the KDE by maximizing an efficient leave-one-out objective. This method optimizes bandwidth using mean LOO log-probability over all observations.

Parameters

xObs	A matrix of observations where each row is an observation and each column is a feature.
labels	A vector of labels corresponding to the observations.
minBw	The minimum bandwidth value for optimization.
maxBw	The maximum bandwidth value for optimization.
algo	The optimization algorithm to use (default is nlopt::LN_SBPLX).
ftol_rel	The relative tolerance for the optimization algorithm (default is 1e-4).
logScaleFlags	A vector of booleans indicating whether to use log-scaling for each bandwidth parameter (default is all false).

◆ objectiveFunctionCV()

double cmp::classifier::KDE::objectiveFunctionCV	(	const method &	method,
		const cmp::statistics::KFold &	kf
	)		const

Objective function for cross-validation. This function computes the objective value for cross-validation, to be maximized.

Parameters

method	The method to use for cross-validation (CV_SCORE or CV_PROB_SCORE).
kf	The KFold object containing the cross-validation splits.

Returns: The objective value for cross-validation.

◆ objectiveFunctionLOO()

double cmp::classifier::KDE::objectiveFunctionLOO ( ) const

Objective function for efficient leave-one-out optimization. This function computes mean LOO log-probability over all observations, to be maximized.

◆ predict()

size_t cmp::classifier::KDE::predict ( const Eigen::Ref< const Eigen::VectorXd > & x ) const

inlineoverridevirtual

Predict the class label at a given point.

Parameters

x	The point at which to predict the class label.

Returns: The predicted class label.

Implements cmp::classifier::Classifier.

◆ predictProbabilities()

std::vector< double > cmp::classifier::KDE::predictProbabilities ( const Eigen::Ref< const Eigen::VectorXd > & x ) const

overridevirtual

Predict the class probabilities at a given point.

Parameters

x	The point at which to predict the class probabilities.

Returns: A vector of class probabilities.

Implements cmp::classifier::Classifier.

◆ set()

void cmp::classifier::KDE::set	(	std::shared_ptr< kernel::Kernel >	kernel,
		std::shared_ptr< cmp::kernel::Bandwidth >	bandwidth
	)

inline

Set the kernel and bandwidth for the KDE. This method allows you to set the kernel and bandwidth objects that will be used for the KDE.

Parameters

kernel	A shared pointer to a kernel object that defines the kernel function.
bandwidth	A shared pointer to a bandwidth object that defines the bandwidth for the kernel.

Member Data Documentation

◆ bandwidth_

std::shared_ptr<cmp::kernel::Bandwidth> cmp::classifier::KDE::bandwidth_ {nullptr}

private

◆ classCounts_

Eigen::VectorXs cmp::classifier::KDE::classCounts_

private

Labels for the observations.

◆ dimX_

size_t cmp::classifier::KDE::dimX_ = 0

private

Input features dimensionality.

◆ kernel_

std::shared_ptr<kernel::Kernel> cmp::classifier::KDE::kernel_ {nullptr}

private

Bandwidth object for KDE.

◆ labels_

Eigen::VectorXs cmp::classifier::KDE::labels_ = Eigen::VectorXs(0)

private

Observations.

◆ nClasses_

size_t cmp::classifier::KDE::nClasses_ = 1

private

Total number of distinct class labels.

◆ nObs_

size_t cmp::classifier::KDE::nObs_ = 0

private

Count of observations per class.

Number of training observations.

◆ xObs_

Eigen::MatrixXd cmp::classifier::KDE::xObs_ = Eigen::MatrixXd(0, 0)

private

The documentation for this class was generated from the following files:

include/classifier.h
src/classifier.cpp

Public Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ KDE()

Member Function Documentation

◆ condition()

◆ density()

◆ fit()

◆ fitLOO()

◆ objectiveFunctionCV()

◆ objectiveFunctionLOO()

◆ predict()

◆ predictProbabilities()

◆ set()

Member Data Documentation

◆ bandwidth_

◆ classCounts_

◆ dimX_

◆ kernel_

◆ labels_

◆ nClasses_

◆ nObs_

◆ xObs_