cmp::classifier::SVM Class Reference

Classifier that uses Support Vector Machine (SVM) for classification. More...

#include <classifier.h>

Inheritance diagram for cmp::classifier::SVM:

Collaboration diagram for cmp::classifier::SVM:

Public Member Functions
	SVM ()
	~SVM ()
void	set (std::shared_ptr< cmp::covariance::Covariance > covariance, const Eigen::Ref< const Eigen::VectorXd > hpar, const double &C, const double &eps=1e-3)
	Set the covariance function, hyperparameters, and SVM parameters. This method allows you to set the covariance function, its hyperparameters, and the SVM parameters C and eps.
void	condition (const Eigen::Ref< const Eigen::MatrixXd > &xObs, const Eigen::Ref< const Eigen::VectorXs > &labels)
	Condition the SVM with observations and labels. This method sets the observations and labels for the SVM and trains the model using the specified kernel.
size_t	predict (const Eigen::Ref< const Eigen::VectorXd > &x) const override
	Predict the class label at a given point.
std::vector< double >	predictProbabilities (const Eigen::Ref< const Eigen::VectorXd > &x) const override
	Predict the class probabilities at a given point.
std::pair< Eigen::VectorXd, double >	getHyperparameters () const
	Get the current hyperparameters and SVM regularization parameter C.
void	fit (const method &method, const cmp::statistics::KFold &kf, const Eigen::Ref< const Eigen::MatrixXd > &xObs, const Eigen::Ref< const Eigen::VectorXs > &membershipTable, Eigen::VectorXd lb, Eigen::VectorXd ub, nlopt::algorithm algo=nlopt::LN_SBPLX, double ftol_rel=1e-4, std::vector< bool > logScaleFlags={})
	Fit the SVM using cross-validation to optimize hyperparameters and C. This method uses the specified optimization algorithm to find the optimal hyperparameters and C that maximize the cross-validation score.
void	fit (Eigen::Ref< const Eigen::MatrixXd > xObs, Eigen::Ref< const Eigen::VectorXs > membershipTable, Eigen::Ref< const Eigen::VectorXd > lb, Eigen::Ref< const Eigen::VectorXd > ub, nlopt::algorithm algo=nlopt::LN_SBPLX, double ftol_rel=1e-4, std::vector< bool > logScaleFlags={})
	Fit the SVM using span-based optimization to optimize hyperparameters and C. This method uses the specified optimization algorithm to find the optimal hyperparameters and C that minimizes the span of the support vectors.
double	objectiveFunctionCV (const method &method, const cmp::statistics::KFold &kf)
	Objective function for cross-validation. This function computes the objective value for cross-validation, to be maximized.
double	objectiveFunctionSpan ()
	Objective function for span-based optimization. This function returns the negative mean support-vector span so it can be maximized.
Public Member Functions inherited from cmp::classifier::Classifier
virtual	~Classifier ()=default

Private Member Functions
void	initDefaultParameters (double C, double eps)
	Initializes default parameters for the SVM model.

Static Private Member Functions
static void	silent_print (const char *s)
	Custom print redirect function to suppress standard console output from LIBSVM.
static void	freeProblem (struct svm_problem *prob)

Private Attributes
struct svm_model *	model_ = nullptr
struct svm_problem	prob_
	Pointer to the SVM model.
struct svm_parameter	modelParameters_
	Structure to hold the SVM problem.
std::shared_ptr< cmp::covariance::Covariance >	covariance_ {nullptr}
	Structure to hold the SVM parameters.
Eigen::VectorXd	hyperparameters_ = Eigen::VectorXd(0)
	Covariance object for the kernel.
Eigen::MatrixXd	xObs_
	Hyperparameters for the covariance function.
size_t	nObs_ = 0
	Total number of training observations.

Detailed Description

Classifier that uses Support Vector Machine (SVM) for classification.

Mathematical Formulation SVM constructs a hyperparameter-tuned decision boundary in a high-dimensional feature space mapped via a kernel:

\[ k(\mathbf{x}_i, \mathbf{x}_j) = \phi(\mathbf{x}_i)^T \phi(\mathbf{x}_j) \]

The dual optimization problem solved is:

\[ \min_{\boldsymbol{\alpha}} \frac{1}{2} \sum_{i,j=1}^N \alpha_i \alpha_j y_i y_j k(\mathbf{x}_i, \mathbf{x}_j) - \sum_{i=1}^N \alpha_i \]

subject to:

\[ 0 \le \alpha_i \le C, \quad \sum_{i=1}^N \alpha_i y_i = 0 \]

where \(C > 0\) is the regularization parameter, \(y_i \in \{-1, 1\}\) are class labels, and \(\alpha_i\) are Lagrange multipliers. Class probabilities are calculated using Platt scaling to compute the probability of class 1:

\[ P(y=1 \mid \mathbf{x}) = \frac{1}{1 + \exp(A f(\mathbf{x}) + B)} \]

where \(f(\mathbf{x}) = \sum_i \alpha_i y_i k(\mathbf{x}_i, \mathbf{x}) + b\) is the decision function, and parameters \(A, B\) are fitted via maximum likelihood on cross-validation outputs.

Implementation Algorithm

1. condition() maps training dataset to LIBSVM nodes, constructs the precomputed kernel matrix \(\mathbf{K}\), and trains the support vectors via Sequential Minimal Optimization (SMO).
2. predictProbabilities() performs Platt scaling by passing computed decision function outputs to Platt's sigmoid model.
3. fit() optimizes hyperparameters (including kernel lengthscales and SVM penalty \(C\)) using cross-validation or span-based bounds.

Constructor & Destructor Documentation

SVM()

cmp::classifier::SVM::SVM ( )

inline

~SVM()

cmp::classifier::SVM::~SVM ( )

inline

Call to LIBSVM function to free the model.

Member Function Documentation

condition()

void cmp::classifier::SVM::condition	(	const Eigen::Ref< const Eigen::MatrixXd > &	xObs,
		const Eigen::Ref< const Eigen::VectorXs > &	labels
	)

Condition the SVM with observations and labels. This method sets the observations and labels for the SVM and trains the model using the specified kernel.

Parameters

xObs	A matrix of observations where each row is an observation and each column is a feature.
labels	A vector of labels corresponding to the observations.

Exceptions

std::runtime_error

if the number of observations and labels do not match.

Functions for the SVM classifier

fit() [1/2]

void cmp::classifier::SVM::fit	(	const method &	method,
		const cmp::statistics::KFold &	kf,
		const Eigen::Ref< const Eigen::MatrixXd > &	xObs,
		const Eigen::Ref< const Eigen::VectorXs > &	membershipTable,
		Eigen::VectorXd	lb,
		Eigen::VectorXd	ub,
		nlopt::algorithm	algo = nlopt::LN_SBPLX,
		double	ftol_rel = 1e-4,
		std::vector< bool >	logScaleFlags = {}
	)

Fit the SVM using cross-validation to optimize hyperparameters and C. This method uses the specified optimization algorithm to find the optimal hyperparameters and C that maximize the cross-validation score.

Parameters

method	The method to use for cross-validation (CV_SCORE or CV_PROB_SCORE).
kf	A KFold object containing the cross-validation splits.
xObs	A matrix of observations where each row is an observation and each column is a feature.
membershipTable	A vector of membership values (class labels) corresponding to the observations.
lb	A vector of lower bounds for the hyperparameters and C.
ub	A vector of upper bounds for the hyperparameters and C.
algo	The optimization algorithm to use (default is nlopt::LN_SBPLX).
ftol_rel	The relative tolerance for the optimization algorithm (default is 1e-4).
logScaleFlags	A vector of booleans indicating whether to use log-scaling for each bandwidth parameter (default is all false).

fit() [2/2]

void cmp::classifier::SVM::fit	(	Eigen::Ref< const Eigen::MatrixXd >	xObs,
		Eigen::Ref< const Eigen::VectorXs >	membershipTable,
		Eigen::Ref< const Eigen::VectorXd >	lb,
		Eigen::Ref< const Eigen::VectorXd >	ub,
		nlopt::algorithm	algo = nlopt::LN_SBPLX,
		double	ftol_rel = 1e-4,
		std::vector< bool >	logScaleFlags = {}
	)

Fit the SVM using span-based optimization to optimize hyperparameters and C. This method uses the specified optimization algorithm to find the optimal hyperparameters and C that minimizes the span of the support vectors.

Parameters

xObs	A matrix of observations where each row is an observation and each column is a feature.
membershipTable	A vector of membership values (class labels) corresponding to the observations.
lb	A vector of lower bounds for the hyperparameters and C.
ub	A vector of upper bounds for the hyperparameters and C.
algo	The optimization algorithm to use (default is nlopt::LN_SBPLX).
ftol_rel	The relative tolerance for the optimization algorithm (default is 1e-4).
logScaleFlags	A vector of booleans indicating whether to use log-scaling for each bandwidth parameter (default is all false).

freeProblem()

static void cmp::classifier::SVM::freeProblem ( struct svm_problem *  prob )

inlinestaticprivate

getHyperparameters()

std::pair< Eigen::VectorXd, double > cmp::classifier::SVM::getHyperparameters ( ) const

inline

Get the current hyperparameters and SVM regularization parameter C.

initDefaultParameters()

void cmp::classifier::SVM::initDefaultParameters ( double  C, double  eps  )

inlineprivate

Initializes default parameters for the SVM model.

Parameters

C	Regularization penalty parameter.
eps	Tolerance of termination criterion.

objectiveFunctionCV()

double cmp::classifier::SVM::objectiveFunctionCV	(	const method &	method,
		const cmp::statistics::KFold &	kf
	)

Objective function for cross-validation. This function computes the objective value for cross-validation, to be maximized.

Parameters

method	The method to use for cross-validation (CV_SCORE or CV_PROB_SCORE).
kf	The KFold object containing the cross-validation splits.

Returns

The objective value for cross-validation.

objectiveFunctionSpan()

double cmp::classifier::SVM::objectiveFunctionSpan

(

)

Objective function for span-based optimization. This function returns the negative mean support-vector span so it can be maximized.

predict()

size_t cmp::classifier::SVM::predict ( const Eigen::Ref< const Eigen::VectorXd > &  x ) const

overridevirtual

Predict the class label at a given point.

Parameters

x	The point at which to predict the class label.

Returns

The predicted class label.

Implements cmp::classifier::Classifier.

predictProbabilities()

std::vector< double > cmp::classifier::SVM::predictProbabilities ( const Eigen::Ref< const Eigen::VectorXd > &  x ) const

overridevirtual

Predict the class probabilities at a given point.

Parameters

x	The point at which to predict the class probabilities.

Returns

A vector of class probabilities.

Implements cmp::classifier::Classifier.

set()

void cmp::classifier::SVM::set ( std::shared_ptr< cmp::covariance::Covariance >  covariance, const Eigen::Ref< const Eigen::VectorXd >  hpar, const double &  C, const double &  eps = 1e-3  )

inline

Set the covariance function, hyperparameters, and SVM parameters. This method allows you to set the covariance function, its hyperparameters, and the SVM parameters C and eps.

Parameters

covariance	A shared pointer to a covariance object that defines the kernel function.
hpar	A vector of hyperparameters for the covariance function.
C	The SVM regularization parameter. Note, this is treated as an extra hyperparameter during fitting.
eps	The stopping criterion for the SVM training algorithm (default is 1e-3).

silent_print()

static void cmp::classifier::SVM::silent_print ( const char *  s )

inlinestaticprivate

Custom print redirect function to suppress standard console output from LIBSVM.

Parameters

s	The string output from LIBSVM.

Member Data Documentation

covariance_

std::shared_ptr<cmp::covariance::Covariance> cmp::classifier::SVM::covariance_ {nullptr}

private

Structure to hold the SVM parameters.

hyperparameters_

Eigen::VectorXd cmp::classifier::SVM::hyperparameters_ = Eigen::VectorXd(0)

private

Covariance object for the kernel.

model_

struct svm_model* cmp::classifier::SVM::model_ = nullptr

private

modelParameters_

struct svm_parameter cmp::classifier::SVM::modelParameters_

private

Structure to hold the SVM problem.

nObs_

size_t cmp::classifier::SVM::nObs_ = 0

private

Total number of training observations.

prob_

struct svm_problem cmp::classifier::SVM::prob_

private

Pointer to the SVM model.

xObs_

Eigen::MatrixXd cmp::classifier::SVM::xObs_

private

Hyperparameters for the covariance function.

Training observation points.

---

The documentation for this class was generated from the following files:

• include/classifier.h
• src/classifier.cpp