The Accelerated CMP Method

Active-learning Gaussian Process surrogates (AS-HGP) for orders-of-magnitude faster modular Bayesian calibration.

While the Complete Maximum a Posteriori (CMP) method resolves the false certitude effect and captures multimodal posteriors without high-dimensional joint MCMC, evaluating the optimal hyperparameter curve \(\boldsymbol{\psi}^*(\boldsymbol{\theta})\) requires solving an internal optimization problem at every MCMC proposal. With each solve scaling as \(\mathcal{O}(N_{\text{hyper}} N_{\text{obs}}^3)\), generating \(10^5 - 10^6\) MCMC samples becomes computationally prohibitive for large datasets or multi-hyperparameter problems.

To overcome this bottleneck, we developed an active-learning surrogate strategy that replaces expensive numerical optimizations with cheap evaluations of a multi-output Gaussian Process, denoted AS-HGP (Active Sampling Hyperparameter Gaussian Process), accelerating calibration by nearly two orders of magnitude.

The Potential Framework & Surrogate Trade-offs

The CMP marginal distribution can be rewritten in potential form:

\[p(\boldsymbol{\theta} \mid \mathbf{y}) \propto p(\boldsymbol{\theta}) \, \exp\left(-\phi(\boldsymbol{\theta} \mid \boldsymbol{\psi}^*(\boldsymbol{\theta}))\right)\]

where the potential \(\phi\) is:

\[\phi(\boldsymbol{\theta} \mid \boldsymbol{\psi}^*) = -\log p(\mathbf{y} \mid \boldsymbol{\theta}, \boldsymbol{\psi}^*) - \log p(\boldsymbol{\psi}^*) + \log \det \mathbf{S}_{\boldsymbol{\theta}}\]

When substituting expensive queries with surrogates, two philosophies emerge:

  1. Log-Posterior GP (lpGP): Approximates the entire scalar potential \(\tilde{\phi}(\boldsymbol{\theta})\) directly. While online evaluation avoids computing the likelihood altogether, it acts as a complete black box, destroying the physics-based likelihood structure and losing direct access to \(\boldsymbol{\psi}\) for subsequent model-error prediction.
  2. Hyperparameter GP (HGP): Builds a surrogate \(\tilde{\boldsymbol{\psi}}(\boldsymbol{\theta})\) for the hyperparameters directly. Although it requires evaluating the likelihood during MCMC, it preserves the residual-based likelihood landscape. Regions of large model residual automatically yield near-zero posterior probability even when surrogate accuracy is imperfect, making HGP dramatically more accurate and robust in high dimensions.

Multi-Output Hyperparameter Gaussian Process (HGP)

Because the hyperparameter mapping \(\boldsymbol{\psi}^*(\boldsymbol{\theta}) \in \mathbb{R}^{N_{\text{hyper}}}\) is multi-output, we construct a correlated multi-output GP by performing Principal Component Analysis (PCA) on the training set \(\boldsymbol{\Psi} = \{\boldsymbol{\psi}^*(\boldsymbol{\theta}_i)\}_{i=1}^{N_{\text{train}}}\):

\[\tilde{\boldsymbol{\psi}}(\boldsymbol{\theta}) = \boldsymbol{\mu} + \mathbf{W} \begin{bmatrix} h_1(\boldsymbol{\theta}) \\ \vdots \\ h_{\tilde{N}_{\text{hyper}}}(\boldsymbol{\theta}) \end{bmatrix}\]

where \(\boldsymbol{\mu}\) is the sample mean vector, columns of \(\mathbf{W}\) are scaled eigenvectors of the hyperparameter covariance, and \(h_j(\boldsymbol{\theta}) \sim \mathcal{GP}(m_j(\cdot), k_j(\cdot, \cdot))\) are independent scalar GPs. The volume correction term \(\log \det \mathbf{S}_{\boldsymbol{\theta}}\) is included as an additional component, avoiding numerical instability in Hessian determinants during runtime.

Active Sampling Strategy (AS-HGP)

Standard space-filling designs such as Latin Hypercube Sampling (LHS) waste training points in low-probability regions of parameter space. Because the posterior support is unknown a priori, we formulate an iterative active learning loop:

1. Posterior-Informed Candidate Sampling

Rather than sampling only from the predictive mean of the GP (which induces severe confirmation bias and premature convergence in early iterations), candidates \(\boldsymbol{\theta}_c\) are drawn across \(M\) distinct stochastic realizations (trajectories) \(\tilde{\boldsymbol{\psi}}^{(\boldsymbol{\omega})}(\boldsymbol{\theta})\):

\[p^{(k)}(\boldsymbol{\theta} \mid \mathbf{y}) = \mathbb{E}_{\boldsymbol{\omega}} \left[ p(\boldsymbol{\theta} \mid \mathbf{y}, \boldsymbol{\omega}) \right] = p(\boldsymbol{\theta}) \, \mathbb{E}_{\boldsymbol{\omega}} \left[ \frac{\exp\left(-\phi(\boldsymbol{\theta} \mid \tilde{\boldsymbol{\psi}}^{(\boldsymbol{\omega})}(\boldsymbol{\theta}))\right)}{\gamma(\boldsymbol{\omega})} \right]\]

Sampling across trajectories broadens the exploration candidate pool wherever surrogate epistemic uncertainty is elevated.

2. Variance-Based Weighted Resampling with Rank-1 Updates

From the candidate pool, \(N_{\text{resample}}\) new training points are selected using importance weights proportional to the variance of the potential:

\[w_{\text{res}}(\boldsymbol{\theta}_i) \propto \mathbb{V}_{\boldsymbol{\omega}} \left[ \phi(\boldsymbol{\theta}_i \mid \tilde{\boldsymbol{\psi}}^{(\boldsymbol{\omega})}(\boldsymbol{\theta}_i)) \right] \approx \left[\boldsymbol{\nabla}_{\boldsymbol{\psi}} \phi\right]^\top \mathbf{K}_{\tilde{\boldsymbol{\psi}}}(\boldsymbol{\theta}_i, \boldsymbol{\theta}_i) \left[\boldsymbol{\nabla}_{\boldsymbol{\psi}} \phi\right]\]

To select batches (\(N_{\text{resample}} > 1\)) without re-training the GPs after every single point, we execute analytical rank-1 updates on the GP covariance conditioned on selected batch points, preventing point clustering and maximizing information gain.

Benchmark Results

Sphere Drag Convergence & Grid Comparison

Evaluating error convergence on the sphere drag benchmark against the exact CMP solution:

  • Convergence: AS-HGP achieves target accuracy with an order-of-magnitude fewer training points than one-shot LHS designs.
  • Support Coverage: The adaptive training grid (black points) clusters precisely along the high-probability posterior manifold (blue contour), whereas LHS (red) scatters samples unproductively across the prior bounding box.
  • HGP vs lpGP: HGP consistently outperforms lpGP at all training sizes because preserving the likelihood prevents catastrophic extrapolation errors.

Structural Design Under Uncertainty

Applying AS-HGP to a structural engineering problem: calibrating the flexural rigidity \(EI\) of an Euler-Bernoulli cantilever beam against data generated by a nonlinear material constitutive law, and optimizing the span \(L\) of a planar frame:

When model error is ignored, the predicted tip displacement is overconfident and excludes the true nonlinear response. With AS-HGP calibration, the 95% confidence interval encompasses the true physical displacement across all design lengths \(L\), confirming robust uncertainty propagation into design.

Publications

  • Journal Article: O. Kahol, O. P. Le Maître, P. M. Congedo, E. Denimal Goy, “Surrogate-based strategies for accelerated bayesian calibration of computer codes with complete maximum a posteriori estimation of model error”, ASME Journal of Mechanical Design, 148(9): 091706, 2026. DOI: 10.1115/1.4071071.